It's the "autonomous" and "swarming" aspect that turn out to be really hard outside of carefully-scripted test cases - and which distract from the more mundane realities of the threat.
Indeed. My semi-stock* answer of "Physics says no" when some of the more fanciful size / range / endurance / payload / on-board processing capability / comms combinations are brought up by VSOs have led to some tetchy conversations.
* - I admit I use it too much. But when I hear someone suggest that we should have electric vehicles and recharge them using solar panels *AT NIGHT*, I feel justified in a long tail of pre-emptive use of the phrase.