You guys are going about this the wrong way...
When someone approaches me and asks "Do you use steroids?"
I look them dead in the eye and ask them "Why do you ask?"
Most of the time they will stammer around and say something like..."Well, because you are really big..."
Then, you look them in the eye again and say, "Thank you for the compliment. I work really hard at being fit."
By that point, they are on the defensive because you have put them on the spot with your answer but because you have just come across as a genuine and appreciative person they are still confused.
That is typically counter to their preconceived expectations of you based on appearance. They expect you to be a jerk or get defensive. Don't. It allows them to rationalize their judgement.
At this point, ask them if they lift and turn the conversation focus on them. I like to use this line....
"So, you lift too, huh? I can tell you have a pretty good build. How much do you bench?"
Again, don't allow yourself to get defensive. Be nice and focus on them. By this time, they have already decided you are a pretty decent guy and will typically say something to make themselves look good to you like "Yeah, I use to lift but I don't get too much time to do it now." Then you can compliment them again and say "Well, you are holding your weight well."
By then they will typically leave and think, "Wow, what a cool guy."
By following that line of conversation, I've never had a person persist with the "Are you on steroids" line of questions and they leave thinking I'm a nice guy.
I do this because I never lie and it works wonders for your perception.