I wouldn't worry about it, they don't advise you not to drink at the clinic while you're taking it and if they were concerned it would make a difference they'd be warning everyone!
I didn't drink when I was on clomid but that was mostly because clomid made me feel so rotten!
Make sure you don't drink in your 2WW - just in case!