Afraid to lose weight

 So this is more of a discussion starter rather than seeking advice.Is anyone else/has anyone else been "afraid" to lose weight due to knowing that people will treat you differently? I feel like it's an unwritten fact that people are treated differently by some depending on their size. I wouldn't say this is holding me back but I feel like I'm almost scared to lose weight because if I start getting treated differently by people around me, it will confirm the fact that people really do judge a book by its cover.I know it's a weird question and I hope I have explained it properly, does anyone else feel like this as well?