Is it an issue of attraction or the fear of being perceived as gay?
If you aren't attracted to him while he's not cross-dressed, you two should have a talk about his plans. If he has no intention of cross-dressing during the week, you should understand that attraction is a large part of a relationship. Most guys wouldn't date ugly women even if it meant they'd get along better.