| Year | Film | Role | Notes |
|---|---|---|---|
| 1941 | Pot o’ Gold | Band member Radio announcer |
Uncredited |
| 1950 | PM Picnic | Narrator | |
| 1964 | The Yellow Rolls-Royce | Joey Friedlander | |
| 1967 | A Guide for the Married Man | Married man “Joe X” | |
| 1974 | Harry and Tonto | Harry Coombes | Academy Award for Best Actor Golden Globe Award for Best Actor – Motion Picture Musical or Comedy |
| 1975 | W.W. and the Dixie Dancekings | Deacon John Wesley Gore | |
| Death Scream | Mr. Jacobs | TV film (aka Street Kill) | |
| Katherine | Thornton Alman | TV film | |
| 1976 | Won Ton Ton, the Dog Who Saved Hollywood | J.J. Fromberg | |
| 1977 | The Late Show | Ira Wells | National Society of Film Critics Award for Best Actor |
| Scott Joplin | John Stark | ||
| 1978 | Movie Movie | Doctor Blaine/Doctor Bowers | |
| House Calls | Dr. Amos Willoughby | ||
| Star Wars Holiday Special | Trader Saun Dann | ||
| 1979 | Going in Style | Al | Pasinetti Award for Best Actor |
| Steel | Pignose Moran | ||
| You Can’t Take It With You | Grandpa Martin Vanderhof | ||
| Sunburn | Marcus | ||
| Ravagers | Sergeant | ||
| 1980 | Roadie | Corpus C. Redfish | |
| Defiance | Abe | ||
| Alcatraz: The Whole Shocking Story | Robert Stroud | TV film | |
| Fighting Back: The Rocky Bleier Story | Art Rooney | TV film | |
| 1981 | St. Helens | Harry Truman | |
| Take This Job and Shove It | Charlie Pickett | ||
| Bitter Harvest | Walter Peary | TV film | |
| The Leprechauns’ Christmas Gold | Narrator / Blarney Kilakilarney | TV film | |
| 1982 | Better Late Than Never | Charley Dunbar | |
| 1983 | The Last Leaf | Mr. Behrman | |
| 1984 | Terrible Joe Moran | Tony | TV film |
| Firestarter | Irv Manders | ||
| The Muppets Take Manhattan | Bernard Crawford | ||
| The Naked Face | Morgens | ||
| The Night They Saved Christmas | Santa Claus | TV film | |
| The Undergrads | Mel Adler | ||
| 1985 | Izzy and Moe | Moe Smith | TV film |
| The Blue Yonder | Henry Coogan | TV film | |
| 1986 | Miracle of the Heart | Father O’Halleran | |
| 1987 | Night Friend | Monsignor O’Brien | |
| 1990 | Where Pigeons Go to Die | Da | Nominated — Primetime Emmy Award for Outstanding Lead Actor in a Miniseries or a Movie |
| 1993 | Last Action Hero | Frank |

