Correct, you shouldn't have if you didn't have a valid defence.
The issue really stems from a lack of definitive case law around the parts of VCRA that're applicable to airsofters. The 'grey area' is only there because a 'valid defence' isn't properly defined in the VCRA. UKARA membership is given as the sole definitive defence, but there are others. Remember, these are 'valid defences' because - by default - it's illegal to buy or manufacture an RIF. You're meant to prove that you're an exception to the rule.
What verb would you like to see in the place of 'manufacture' that stops people from just spraying their blue gun black? Because the spirit of VCRA is certainly such that if you're swapping parts at home that's considered manufacture.
If I have a factory built where all we do is take CM-16s and put ACOGs on them, what verb do you use then? Is that manufacture? I would argue to all intents an purposes that it probably is, because that's less confusing than granularity going into every way in which something is and isn't manufacture. Paint wearing off probably isn't manufacturing either, so here we see where that hastily added amendment to VCRA is clearly not fit for purpose. I know I can buy every part of an RIF and assemble it at home (now that's clearly manufacture, so how is swapping handguard and stock not when the only thing making it an IF - rather than an RIF - is that very handguard and stock)?
However, it is actually the only thing we have that allows us to even buy RIFs in the first place so I'm not sure if brazenly suggesting that someone just 'buys 2-tone and spray black' is the best course of action for the longevity of airsoft. If your problem is with the verbiage in the law then I'd urge you to read it in context.