In the late 19th century, a "ronin" (Toshiro Mifune) (a samurai without a master) wanders down a dusty path. Happening onto a house, he asks the man to drink from the man's well. The man is busy berating his teenage son, who is going off to join a nearby gang of gamblers and doesn't want "a long life of eating gruel". After the son leaves, the man insults the ronin by saying that all sorts are attracted when "they smell blood".
Walking into the nearby town, the ronin sees the streets are empty. A few angry glances from young gang members (mangier even than the ronin) are shot at him from sliding doors. The ronin is caught-off guard when he sees a dog trot by with a human hand in it's mouth. A young-looking gang confronts the ronin, but he wanders off unfazed.
He wanders into an empty tavern and orders to the eldery owner, Gonji (Eijirô Tono). Gonji only has hot saki and cold rice, but the ronin hungrily engulfs it. Gonji explains the hiearchy of the town to the ronin. Seibei (Seizaburô Kawazu) is the older crime boss of the town, and runs a brothel with his domineering wife, Orin (Isuzu Yamada). Seibei appointed Tazaemon (Kamatari Fujiwara) the town's major, but Tazaemon is like a puppet for Seibei. The competing and more powerful crime boss is Ushitora (Kyu Sazanka), who rules with violence along with his pig-like brother, Inokichi (Daisuke Katô). Ushitora has appointed the corrupt sake-brewer Tokuemon (Takashi Shimura) as the mayor (also) and is more widely-recognized than the powerless Tazaemon. Thinking that his explaination of the town would motivate the ronin to leave, Gonji is shocked when the ronin says that he's going to "stay for a while".
The ronin goes over to Seibei's home and summons his gang out. He tells them he's going to show them his skills. The ronin then goes over to Ushitora's home and is confronted by the same hoodlums who surrounded him earlier. As they posture in front of him, the ronin promises to kill them. The hoodlums attack and the ronin kills several of the men in a matter of seconds. The ronin tells the nearby cooper (Atsushi Watanabe) how many coffins he needs to make.
The ronin returns to Seibei's home and Seibei gratefully accepts him. Since he is planning to attack Ushitora, Seibei wants to employ the ronin as a bodyguard. The ronin negotiates the salary to a much higher amount and agrees to the position. Seibei asks the ronin was his name is and, after the ronin puzzles for a moment out the window at a mulberry field, the ronin says his name is "Sanjuro Kuwabatake" (Thirty-something mulberry field) although he says he's "actually nearly forty". Orin comes and calls Seibei to the other room. Sanjuro evesdrops on them and hears that they're planning to kill him after they use him to lead the attack against Ushitora.
At noon, Seibei's men and Sanjuro gather outside across the street from Ushitora's gang. When they start marching forward, Sanjuro walks off. He let's Seibei know that he overheard their scheme to kill him. Sanjuro climbs the nearby belltower and laughs as he watches the two cowardly gangs lunge and retreat at each other on the street. The gangs never make contact and are interrupted when an inspector comes inquiring about an official killed nearby (actually killed by Ushitora's men). Ushitora and Seibei promise to finish their clash later.
Sanjuro return's to Gonji's tavern and watches out the window Ushitora and then Seibei fawn over the inspector. Later, Ushitora and his giant thug, Kannuki (Namigoro Rashomon), come into Gonji's tavern and offer Sanjuro a position with him as a bodyguard. Sanjuro says he has many offers and has to think it over. Later still, when the wind is blowing gusts, a commotion surrounds the return of Unosuke (Tatsuya Nakadai), Ushitora's youngest and deadliest sibling. Unosuke has attained a pistol in his world travels and shows off with it by shooting at the town bell.
That night, Sanjuro convinces Seibei that he will spy on Ushitora. As a gratitude, Orin offers her geishas to Sanjuro but he tiredly declines. Out on the street, Sanjuro happens upon two of Ushitora's drunken thugs. He incapacitates them with his sword and ties them up. He goes back to Ushitora and says he caught the two conspiring with Seibei. In response, Unosuke gleefully shoots both of the men. As a further strike back, Seibei's son Yoichiro is taken hostage by Unosuke and Ushitora's thugs.
Shortly thereafter, as Sanjuro watches from Gonji's window, the two gangs meet in the street. A young man, Kohei (Yoshio Tsuchiya) and his young son (Yosuke Natsuki), come into Gonji's tavern and watch anxiously. Ushitora tells Seibei that he must hand over his territory for Yoichiro. Seibei reveals he has a bargaining chip as well, the beautiful Nui (Yôko Tsukasa). Nui is Kohei's wife, but after he came to owe gambling money to Ushitora, Nui was pimped out to Tokuemon. Kohei has had to listen as Tokuemon ravished Nui every night. He is also regularly beaten by Inokichi. When the two gangs agree to trade their hostages, a tense situation ensues as Nui realizes her son and husband are at Gonji's. Nui is dragged away from her family and back to Tokuemon's and Ushitora's clutches.
That night, Sanjuro finds out where Nui is being kept and that six men are guarding her now. Sanjuro walks along with Inokichi and reminds him that he needs to beat Kohei. Shortly after Inokichi goes into to do so, Sanjuro rushes and tells Inokichi that the six guards have been killed and that he must go back and tell Ushitora and Unosuke. When Inokichi rushes off, Sanjuro burst in on and kills all six guards quickly. He burst in on and pulls out Nui and reconnections her with Kohei and their son. Sanjuro trashes the room to make it look like a struggle has occurred. When he returns outside, Sanjuro finds Kohei & Nui tearfully bowing to him, much to his annoyance. He says he hates pathetic people and tells them he will kill them if they don't run, which they promptly do.
Seconds later, Ushitora, Unosuke and some of their thugs come back and are convinced that the massacre of the guards is the act of Seibei's men. Shortly thereafter, Tokuemon's sake brewery is destroyed with several well-placed slashes. Ushitora is again convinced this a strike by Seibei, but Unosuke begins to suspect another source of the skillful sword-marks left behind at these attacks.
Sanjuro walks into Gonji's tavern and is disconcerned to see the usually grumpy old man beaming. It turns out Kohei has left a thank you note for rescuing them and Gonji is happy that Sanjuro is a good man underneath the gruff and violent exterior. However, Unosuke enters the tavern and begins to grill Sanjuro. Unosuke is able to take away Sanjuro's sword and intercepts Gonji as he tries to take off Kohei's note.
Sanjuro is captured and beaten badly by Kannuki and the rest of Ushitora's gang. Despite the continual beatings, Sanjuro refuses to disclose Nui's location (which he actually doesn't know). During a break in the beatings, the badly-injured Sanjuro hides in a chest in the room. Kannuki and a cohort return and are bewildered that he's not there. When Kannuki rushes back to tell the others (leaving the previously locked door open), Sanjuro crawls outside and gets under a nearby walkway just in time. Ushitora's whole gang desperately search for Sanjuro, not realizing he's literally right under their feet. Seeing an opening, Sanjuro manages to crawl on to the street and to Gonji's tavern. Sanjuro tells Gonji to tell Ushitora's men that he went to Seibei's and hides in the corner of the tavern under some debree.
When Gonji tells Unosuke Sanjuro's story, Unosuke goes back to Ushitora and pitches this situation as an change to finally iradicate Seibei and his men. Meanwhile, Gonji gets the cooper to place Sanjuro in a coffin so they can carry it off to a secure location. When Gonji and the cooper carry out Sanjuro in the coffin, he asks them to stop so he can peak out and view the chaos as Ushitora finally eliminates Seibei. Ushitora's men have set Seibei's brothel on fire, forcing all of Seibei's out to shot by Unosuke or slashed to death by Ushitora's men. Finally, Seibei, Orin and their son come out surrendering. Despite being unarmed, they are brutally killed by Unosuke and Inokichi. While they watched the ghastly spectacle, the cowardly cooper has ran off. Gonji is forced to convince the now lone Inokichi to help him carry the coffin containing Sanjuro. Gonji and Inokichi carry Sanjuro to the graveyard, but Gonji dupes Inokichi into going back into town to look for Sanjuro. Sanjuro takes refuge in a nearby temple.
After several weeks of recovering, Sanjuro shows his returned skill by catching a floating leaf on a knife he throws. When Sanjuro hears somebody coming, he thinks it's Gonji coming to bring him more food. But it's actually the cooper and he tells Sanjuro that Ushitora's men captured Gonji carrying food and are beating him for Sanjuro's location.
Angrily, Sanjuro arms himself with a sword and a knife and marches into town. A beaten Gonji is seen hanging prostrate from a rope in the middle of town. Realizing that Sanjuro is there, the whole of Ushitora's gang gathers in the street. Sanjuro menacingly stomps towards them. Unosuke pulls out his pistol and warns Sanjuro to stop. Sanjuro instead breaks into a sprint. Unosuke levels his gun at Sanjuro, but Sanjuro throws his knife into Unosuke's arm before he can even fire. With amazing speed, Sanjuro unsheathes his sword and lethally slashes Ushitora's whole gang, including Unosuke, Inokichi and Ushitora himself. Sanjuro spares only one of Ushitora's cowering thugs, recognizing him as the son of the house when he wondering into the town. He tells him to "live a long life eating gruel!" Lying in a puddle of his own blood, Unosuke asks Sanjuro to hand him his pistol because he feels naked without it. When Sanjuro gives Unosuke his pistol, Unosuke grins and points it at Sanjuro, pulling the trigger. The gun is empty and Sanjuro doesn't flinch. Unosuke then dies. A crazed-looking Tazaemon suddenly stumbles across the street and kills Tokuemon. Sanjuro cuts down Gonji from the rope and tells him that the town should be "pretty quiet" now. Sanjuro says "See ya later" and walks off out of the now nearly empty town.