It sounds like they are ..well..nevermind..I'm sure I can't say that, but you know what I mean!
You seem like a nice person who treats everyone fairly no matter what their skin color is and that's cool! Don't change that about yourself!
Don't worry about what people say about you cause of who you hang out, cause they people you hang out with are obviously better friends tham the ones that judge you!
Since that the bullying case is getting worse, talk to an adult, like your un-racist and trustful teacher or principal there, and your parents!
Just trust her and your instincts and believe in yourselves.