Before becoming an actor O'Connor served in the United States Merchant Marine during World War II. He made his acting debut as a character actor on two episodes of Sunday Showcase. From there he went on to star in such shows as Gunsmoke, I Spy, The Man From U.N.C.L.E. and The Eleventh Hour. Later in life he starred in the hit comedy series Mad About You, next to Paul Reiser and Helen Hunt.

O'Connor's long running career ended on June 21, 2001, when he passed away after suffering from a heart attack.

1. "Nothing will give me any peace. I've lost a son. And I'll go to my grave without any peace over that." — Carroll O'Connor on his son's death
2. "It was a lack of system that made the 30's Depression as inevitable as all others previously suffered." — Carroll O'Connor on the Depression
3. "Get between your kid and drugs, any way you can, if you want to save the kid's life." — Carroll O'Connor's message to kids on drugs
4. "I never heard Archie's kind of talk in my own family. My father was a lawyer and was in partnership with two Jews, who with their families were close to us. There were black families in our circle of friends. My father disliked talk like Archie's -- he called it lowbrow." — Carroll O'Connor on his own family