An open smile was seen as low-class
Etiquette books like Castiglione's The Book of the Courtier taught that baring your teeth was vulgar, fit for peasants and drunks. A grin would also have made the whole portrait about the smile, which no paying sitter wanted. Bad teeth had little to do with it.
