People think I'm racist..?

I was hanging out with some girls I work with at a local grocery store and we were making jokes about each other. My jokes were about the extremely fair shade of my skin, my somewhat red hair and my glasses. The other girls were teased about everything from their hair texture to their weight. It has started to rain so we were joking around with the black girl in the group about how she couldn't get her hair wet. She laughed along with us saying she needed to "get her hair did" and we continued on. We joked again the next day about similar topics and everything seemed fine. However, it turns out that she had told a coworker that I was being racist and had offended her with no regard to her feelings. I understand that some things said may be perceived as racist, but the problem here is that she did not tell any one of us who were making the jokes. Instead, she involved someone who was not there and did not know what happened besides what they were told(and, on top of that, this girl was messier that most freshmen I know). The next day, I got called into the boss' office and even though, I did not get in trouble because he understood my point, I'm being harassed by my coworkers. One of the guys in the meat department(who is black) that I used to talk to and get along with really well now hates me. I'm also concerned about what this girl and the other cashier she involved is telling people. For all I know, they could be portraying me as a KKK member. I don't think that reporting them to the boss would do any help because there's at least five people and it seems like they're using this number of people to out number me and the people who will stick up for me as a way to bully me into quitting. I don't understand how this situation got this way and I need advice on how to deal with it because it doesn't seem like ignoring it will help :/