The idea that you can say that, because most people found mahts their least-favourite subject at school means that there is more to music than maths is - well, I've covered this already. High-school maths isn't maths, it's just a collection of numerical methods that you might find useful at a supermarket or building a child's playpen. That's not maths, in any way, shaper or form. If you want to know what maths is, study it at a top university. If you pass through that and still find maths uninteresting, then of course that's your opinion and there's nothing wrong with it. But I'd advise you to talk to mathematicians rather than the man in the street.
By the same token, would you say that if you went up to someone and explained to them that, "a computer takes a series of binary square-wave inputs induced by components that possess the properties of resistance, capacitance and inductance (in the case of a square-wave, the capacitor will dominate this circuit if connected to an operational amplifier that is cyclically overloaded) and by passing this square-wave input through a series of silicon chips and logic gates produces a square wave output that, in turn, passes through another circuit to produce a characteristic monitor response according to the American Standard Code for Information Interchange..." that people would suddenly stop using computers? Admittedly, while much of the above paragraph is essentially made up, that's only because I've simplified it. The people you told that to would give you a funny look and then say, "Yeah, but do I need to now that?" and then carry on as normal.
A bit like I'd expect them to if you told them that, "the notes on a standard treble-clef stave represent the approximately linear relationship between the height of the note on the stave and its frequency when mapped to logarithmic space." Which, for the record, is true. There's no way that anyone is seriously going to stop listening to music because you tell them it's all maths because, frankly, they wouldn't believe you. And probably wouldn't care either.
As a final aside, why does it make any difference to the truth of the matter if everyone thinks otherwise? E.g "the earth is flat", "heavier objects fall faster than light ones", "Cliff Richard is a good singer" and so on.
