Honestly, you scooter. You're going to get shit from pretty much anybody who's ever been annoyed by another kid on a scooter. It's a fact of scootering.
Best thing to do is get really REALLY good at scootering, so when someone goes to shit on you, you whip out tricks that blow their fucking minds.
Realizing that most of the kids that hate scootering, actually suck at their sports, is another thing that'll bolster your confidence.
You could also do what I did and get moderately good at every sport. I'm a really good inliner, I can skateboard... a little bit, and I can flow bikes with no problem. This way when they attempt to shit on you, you can ask them for their equipment (for lack of a better term), ride it, and then they'll have no reason to shit on you anymore.
Also, search. If I had a dollar for every "what to do about haters" thread I'd be rich.