If I had to go back to origins of things, then I'd consider "master" to be the male opposite of "mistress", in many cases.
Picture a rich man's slave back in the 1700's. They're given an order. How do they respond?
To the lord of the estate: "Yes, master." Or maybe, "Yes, Milord," if one wanted to sound grandiose and earn browny points flattering the follow.
For the guy's wife? "Yes, mistress." Or, "Yes, Milady."
Even in modern BDSM circles, I'd think the opposite of a "mistress" would be a "master".
So, like most words we use, I imagine any counterpart to it would have to be defined by the *exact* definition one was applying to the original word in question.
Mistress as a man's lover? The male opposite to that is an inamorato or paramour.
Mistress as a dominant controller? The male opposite to that is a master.
If you're in an area with British English prevalent, then a mistress is a female teacher. Or tutor. I suppose the opposite for them might be "professor", but I wouldn't swear to it. There might be a closer male approximation to the usage than that, that I'm just overlooking.
:P
(As you can probably tell by now, I'm a language/word nut. LOL!)