Davis began his career at the age of 3 with black vaudeville troupe Holiday in Dixieland helmed by his defacto uncle Will Mastin. Davis proved to be so popular that the act was renamed Will Mastin's Gang Featuring Little Sammy and eventually the Will Mastin Trio. In the late 40s and early 50s, the Trio played many of New York and LA's biggest nightclub venues and were invited by Frank Sinatra to play the newly-integrated Copacabana in 1953.

In 1954, Davis signed a solo record deal with Decca. Shortly thereafter, he lost his left eye in a much-publicized automobile accident.

By 1959, Davis had become associated with Sinatra's Rat Pack and was regularly performing at the Sands in Las Vegas with Sinatra and Dean Martin. Davis appeared in dozens of films at the time including Robin and the 7 Hoods and Ocean's Eleven with Sinatra.

Throughout his career, Davis' popularity helped integrate nightclub venues, casinos and hotels where, prior to his appearance, black entertainers could perform, but not stay.

Davis died of throat cancer in 1990 at the age of 64.