In voiceover, Irish-American mobster Frank Costello (Jack Nicholson) explains that he doesn't want to be the product of his environment, he wants his environment to be a product of him. Grainy historical film depicts violent riots in Boston during earlier generations. Costello, proudly Irish, unapologetically racist, explains that what annoys him about African Americans is their refusal to realize that they won't be given anything. Costello's belief is you have to do whatever possible to take what you want.
Costello is in a small convenience store and collects protection money from the shop owner, a man visibly unnerved by Costello. Costello warns him to have more money next time. Costello asks the shop owner's young teenage daughter behind the counter if she's got her period yet. His attitude is nonchalant and all the more threatening for it. The daughter is both affronted and compelled by the powerful Costello and answers his rhetorical question. The shop owner doesn't show any emotion.
A small boy, a young Colin Sullivan (Conor Donovan), witnesses Costello's interaction with the shop owner. Costello notices him and asks him if he's the son of a man Costello knows. He tells the store owner to give the boy two full bags of groceries and then presses some coins into the boy's hand. He tells the boy to remember he can get more money from Costello when he gets older.
As Sullivan comes up in age, Costello grooms him and steers him into a job as a police officer, where he can keep Costello informed. Sullivan graduates from the police academy and is immediately transferred to the Massachusetts State Police force. He is accepted into the Special Investigations Unit led by Ellerby (Alec Baldwin) which focuses on organized crime. The primary target of the unit is Costello. Sullivan (Matt Damon) is warmly welcomed onto the State Police force by Captain Queenan (Sheen) and Staff Sergeant Dignam (Wahlberg).
Billy Costigan (DiCaprio), whose extended family has considerable ties to organized crime, also attends the police academy. Before he graduates, he is summoned to a meeting with Queenan and Dignam who intimidate, bully, and verbally harass him. They pressure him to disqualify himself as an officer because he's too smart to be a cop and because of his family's ties to crime. When Costigan stands up to their hazing, they change tactics: they offer him a job as a police officer on one condition: that he quit the academy, serve serious time in jail on a trumped-up assault charge, and infiltrate Costello's organization. Only Queenan and Dignam will know that Costigan is working undercover. They promise him a bonus when he completes the assignment.
When Costigan gets out of prison, he contacts his cousin and tries to finance a drug deal. Costigan and his cousin meet Mr. French (Ray Winstone) at a local bar, where Costigan orders a cranberry juice. The guy on the bar stool next to him asks Costigan if he is having his period. Costigan smashes a beer mug over the man's head and is halted Mr. French. Mr. French warns Costigan that he is not allowed to beat up certain men, and he's lucky the guy at the bar is not one of them.
Eating at the counter of a store like that in which Sullivan originally met Costello, Costigan confronts two Italian mobsters from Providence extorting protection money from the store owner. He beats the two men up badly, breaking his hand when he hits one of them. Costello learns of Costigan's actions and summons him to a meeting. He offers to protect Costigan from the Mafia, who, he promises, will return with reinforcements to kill Costigan. He invites Costigan into a back room where he instructs Mr. French, a senior member of Costello's crew, to check Costigan for weapons or a wire. Mr. French breaks Costigan's cast open and Costello beats Costigan's broken hand with a shoe, trying to get him to confess being a cop. Costigan withstands their beating and is accepted as a legitimate crook by Costello.
Both Sullivan and Costigan gain credibility within their respective organizations. Sullivan visits a crime scene where the two Mafia men from Providence are found. Sullivan contacts Costello, who directs him to influence the investigators away from Costello.
Sullivan begins dating psychiatrist Madolyn Madden (Vera Farmiga). Costigan also sees her but as a client, required under the terms of his probation. He also develops an attraction to her. During a deal to sell stolen missile guidance microchips to Chinese government agents, Sullivan warns Costello of the police operation and everyone avoids police detection by boarding boats waiting at the back of the warehouse.
It becomes evident to both Sullivan and Costigan that there is a mole in each other's organization, though their identities cannot be determined. Costigan talks to Dignam and threatens to leave but Dignam tells him to come up with evidence so they can find the mole. Extremely agitated, Costigan relents. He has coffee with Madden who plans to move in with Sullivan.
Costello tells Sullivan to find the "rat" among his crew. Sullivan asks for his crew members' social security numbers and other forms of ID so he can track them down. Costigan searches for the informant and learns from a member of Costello's crew that Costello himself is an FBI informant, explaining why federal prosecutors repeatedly fail to indict and arrest Costello. Costigan visits Queenan at home late at night and tells him that Costello is an FBI informant.
Mr. French collects everyone's social security numbers and other ID. Costigan corrects Fitzgibbons's (David O'Hara) misspelling of 'citizens' on the envelope containing their information and then leaves. He visits Madden at her apartment, where she is finishing moving out. She tells him she is moving in with Sullivan. They talk and then have sex.
Ellerby puts Sullivan in charge of the investigation to find the mole in the Special Investigations Unit, citing his "immaculate record." Costigan follows Costello to a porn theater, where he sees Costello meet with and give a dark figure -- unknown to him, Sullivan -- the envelope containing Costello's crew members' personal information. Costigan chases Sullivan out the emergency exit of the theater and through Chinatown but neither man learns the other's identity.
Sullivan tells Costello to trail Queenan to a meeting with Costigan. Costigan gets away before Costello's men throw Queenan off the roof, landing at Costigan's feet. As Costello's crew leaves, Costigan joins them, pretending he's just arrived to join them in the assassination. Delahunt (Mark Rolston) is mortally wounded. Back at their bar, Delahunt fingers Costigan as the mole but dies before he can tell anyone else.
A news report reveals that Delahunt, a crew member, was an undercover cop. Consequently, Dignam is forced to step down from the police force.
Using Queenan's phone, Sullivan reaches Costigan, and fails to persuade him to quit his work as a mole. Sullivan learns from Queenan's diary that Costello was an informant for the Federal Bureau of Investigation. He worries that his identity as a mole for Costello may be revealed. With Costigan's help, Costello is traced by the police to a cocaine pick-up, where a gunfight erupts between his crew and police, during which most of Costello's crew are killed. Sullivan confronts the wounded Costello, who admits he is an occasional FBI informant. Sullivan shoots him multiple times. Sullivan is applauded the next day for having killed Costello by everyone on the force.
In good faith, Costigan comes to see him, seeking to get his civilian identity restored and to collect his back pay. He tells Sullivan he intends to resume his civilian life. Sullivan leaves to look up Costigan's employee record when Costigan notices the envelope from Costello on Sullivan's desk. Costigan finally realizes Sullivan is Costello's mole. Returning to his desk, Sullivan realizes that Costigan has figured out his true identity, so he erases Costigan's employee records from the police computer system.
Madolyn tells Sullivan she's pregnant, but doesn't reveal who the father is. A few days later she receives a package in the mail from Costigan addressed to Sullivan. She opens it to find it contains a CD of Costello's recorded conversations with Sullivan. Sullivan walks in as she is listening and tries unsuccessfully to assuage her suspicions. He contacts Costigan, who reveals that Costello recorded every conversation he had with Sullivan. Costello left the recordings with his attorney, who has given them to Costigan. Costigan says he wants his civilian identity back or he will implicate Sullivan. They agree to meet on the roof of the same building where Queenan was killed.
When they meet, Costigan catches Sullivan off-guard and handcuffs him. As Costigan had secretly arranged, Officer Brown (Anderson) appears on the roof as well. Shocked to see Sullivan in handcuffs and held at gunpoint by Costigan, Brown draws his gun on Costigan. Costigan explains his actions by revealing that Sullivan is the mole. Costigan asks Brown why Dignam did not accompany him, but Brown doesn't answer. Costigan leads Sullivan to the elevator and Brown takes the stairs to follow them.
When the elevator reaches the ground floor and the doors open, Officer Barrigan (Dale) shoots Costigan in the head. When Brown arrives, Barrigan kills him too. Barrigan reveals to Sullivan that Costello had more than one mole in the police and that Costello was going to give both of them up to the FBI. When Barrigan momentarily turns, Sullivan shoots him in the head. At police headquarters, Sullivan concocts a story to protect himself, identifying Barrigan as the mole and recommends Costigan for the Medal of Merit.
At Costigan's funeral, Sullivan and Madolyn stand by the grave. Sullivan attempts to talk to her, but she ignores him. As Sullivan enters his apartment, he is met by Dignam, who shoots and kills him. As the frame drifts up to the window and the gold dome of the Massachusetts State House in the background, a rat scurries along the balcony railing.