Born in Seattle, Washington, in 1922, as Solomon Krakovsky, to Russian Jewish immigrants, Hill became interested in the theater as a little boy. After graduation from high school, he served in the Naval Reserve (1940-44). From there, he worked alongside a young, unknown star Marlon Brando in "A Flag is Born." His real acting debut came about in 1950, when he co-starred opposite Hedy Lamarr in A Lady Without Passport (1950), and his career took off from there.
For the rest of the 1950s and 1960s, he co-starred in B-movies such as: The Goddess (1958), Kiss Her Goodbye (1959), A Child Is Waiting (1963), The Slender Thread (1965). Hill also became a leading character actor guest-starring in a wide variety of shows such as: Schlitz Playhouse (1951), Danger (1950), Playwrights '56 (1955), Studio One in Hollywood (1948), Playhouse 90 (1956), Naked City (1958), Espionage (1963), Dr. Kildare (1961), Ben Casey (1961), among many others. Another guest-starring role on Rawhide (1959), led him to starring on Mission: Impossible (1966). Though the series lasted seven seasons, Hill was not pleased with his role because he refused to abide by the production schedule that required working on Saturday, the Jewish Sabbath, being an Orthodox Jew. Hill was fired after 1 season, and was replaced in the series by Peter Graves. After his firing, he went on a sabbatical from acting, to focus on selling homes in real estate, hence he moved to New York.
After his long absence from acting, he returned to the small screen with the TV mini-series, King (1978). The 1980s saw Hill returning to the box office, co-starring in such blockbuster films such as: It's My Turn (1980), opposite Jill Clayburgh and Michael Douglas, Eyewitness (1981), Yentl (1983), Garbo Talks (1984), opposite Anne Bancroft and Carrie Fisher, On Valentine's Day (1986), and it's sequel, Courtship (1987), he played a Mafia Don in Arnold Schwarzenegger's another blockbuster film, Raw Deal (1986), then, in a blockbuster ten movie Running on Empty (1988), followed by The Boost (1988), White Palace (1990), and one of the last films, Billy Bathgate (1991).
In the 1990s, at 68, after a 23-year-absence from the small screen, he reestablished his career, as the last minute replacement for a starring role in Law & Order (1990), where he played the role of a pragmatic District Attorney, Adam Schiff, a character whom everyone grew to love. Despite not appearing in the pilot episode of the series, he quickly became one of Hollywood's likable and bankable stars, where he often had scenes, that were filmed, only in court. His co-stars on the show consisted of actors who have also made bankable names like Hill himself like Chris Noth, former Pee-wee's Playhouse (1986) alumnae, S. Epatha Merkerson, Sam Waterston, and newcomers Jill Hennessy and Benjamin Bratt. On Law & Order (1990) he was also nominated for Emmies twice, but did not win. The entire cast was shocked when, in 2000, he left his role to enjoy his retirement. Before then, the entire cast appeared with Hill on Larry King Live (1985), to say goodbye to a legendary star.