Set in 1937 Los Angeles, a private investigator named Jake "J.J." Gittes (Nicholson) is hired to spy on Hollis Mulwray, the chief engineer for the city's water department. The woman hiring Gittes claims to be Evelyn Mulwray, Hollis' wife. For unknown reasons, Mulwray spends considerable time looking at dry riverbeds and water outlets. Mulwray is photographed while having a heated argument with an elderly man on the street. Gittes photographs Mulwray from a roof top when he kisses a young blonde, and the photo is published in the newspaper the next day, causing a scandal. After the story is published, Gittes is visited by the real Mrs. Evelyn Mulwray, who threatens to sue him for defamation.
Gittes uncovers information that despite a serious drought and an expensive proposal to build a new dam, the Water and Power Department is dumping fresh water into the ocean at night. The dam project is opposed by Mulwray himself, who cites a potential disaster because of weak geological formations in the rock where the dam is to be constructed -- Mulwray had previously supported the building of another dam that had failed due to similar geological conditions. When he addresses a public hearing on the project, which Gittes attends, Mulwray is ridiculed by several farmers, one of whom leads a flock of sheep into the room, who want the reservoir and water the dam will provide. Gittes also meets with Mulwray's successor, Yelburton, who tells Jake that the Dept. of Water has been diverting some of the water supply to farmers in the San Fernando Valley because of the drought. He says specifically that there's always some run-off into various smaller waterways and drain channels.
Jake goes to Mulwray's mansion to speak to him but is only able to talk to Evelyn Mulwray. While he waits for her, Mulwray's Japanese gardener cleans a small decorative pond, pulling out a clump of grass and tossing it aside. He casually says "bad for the glass" in broken English, a comment that Jake dismisses. Jake notices a shiny object in the pool and tries to retrieve it, stopping when Evelyn appears. She tells him that Mulwray usually takes afternoon walks at a reservoir and that he should look for him there.
Gittes goes to find Mulwray at the reservoir -- bluffing his way past the checkpoint with one of Yelburton's business cards -- but instead finds the police and firemen hauling Mulwray's body out of the reservoir. They believe Mulwray died from drowning. Gittes used to work with the lead investigator, Lt. Lou Escobar. When the police interview Evelyn Mulwray about her husband's death, they assume she hired Gittes, and Gittes corroborates the lie for her. She thanks him and hires him to investigate what happened to her husband. Later, at the county coroner's department, Gittes learns from the coroner that Mulwray's lungs were filled with salt water, although he had been found in a freshwater reservoir. Gittes also learns that a local drunk was found dead from drowning in a dry riverbed that Mulwray had been inspecting.
Later that night Gittes climbs the secure fence around the reservoir and, after he is shot at by an unseen shooter, is nearly drowned when the culvert he is hiding in has a sudden rush of fast-moving water. Two water department security thugs detain him; a large man named Claude Mulvihill and a short thug (a cameo by Roman Polanski), who slashes Jake's nose for being a "very nosy fella." They threaten to cut off the rest of his nose if he comes snooping around again. Gittes, forced to wear a large and ridiculous bandage, receives a call from Ida Sessions, the woman who originally impersonated Evelyn Mulwray. She admits she was hired to trick Gittes, but refuses to come to his office. She tells him to read that day's obituaries, saying he'll find "one of those people". At the water department, Gittes sees photographs of the elderly man Mulwray quarreled with a few days before his death; the man is Noah Cross (John Huston), Evelyn Mulwray's father. He used to own the water utility in partnership with Mulwray, but Cross ended his association with the department when the partners sold it to the city, as Mulwray had long desired. He speaks briefly with Yelburton again and tells him that he suspects Mulwray was murdered. Yelburton is incredulous and Jake tells him he doesn't want to see Yelburton accused or charged with the crime but wants to nail the people behind it.
Cross invites Jake to lunch at his home and hires Jake to find the blond girl Mulwray had been seeing, saying that she might know what happened to him and that he'd like to comfort her if he can. Cross hints that Jake might be sleeping with Evelyn in addition to gouging her in his investigation of Mulwray's murder but relents when he sees Jake's resolve.
Gittes goes to the Hall of Records and looks in a large plat book of the valley. He learns that a considerable portion of the valley has been bought up in the past few months by a few new land owners who have purchased large tracts of land. When the attendant in the room refuses to let Jake borrow the book, Jake surreptitiously tears the column out of the book and pockets it.
Acting on another tip from Sessions, Gittes begins to unravel an intricate scandal involving LA's fresh water supply. Gittes first travels to an orange farm to talk with the owner about how his land is being irrigated. As he drives around he is shot at by the farmer and a few of his farmhands and crashes his car into a tree. Jake is dragged from his car, beaten and searched. The farmer explains that the Department of Water & Power has been harassing him by sending agents to run him off his land and poison the water in his wells, suspecting that Jake might be one of them. While he tries to show the farmer documentation of his investigation, the farmhands claim that Mulwray is responsible for harassment of late and attack Jake. When Jake tries to fight back, he's knocked unconscious. He wakes up to find that the farmer and his wife have called Evelyn, who has come to the farm.
While they drive back to LA, Jake explains the scandal to her: her father and his partners have been forcing farmers in the rural areas surrounding the city off their land so they can buy it cheap, after which a newly-built (and controversial) dam and water system would start redirecting much of L.A.'s water supply to that land, dramatically increasing its real estate potential and value.
Since Cross wants no record of such transactions, he has partnered with a retirement home community, using the identities of the eldest residents within (one of whom is mentioned in the obituary column): they would legally, but unknowingly, own the land. Jake, having matched one of the obituary names to one of the names in the list he stole from the plat book, has Evelyn drive him to the retirement home where they pose as a married couple trying to find a place for Jake's father to live. The host tells them they can tour the facility. They come across an activities board with the names of the people from the plat book. Jake talks to a group of women working on a quilt. One of the pieces of fabric they've sewn into the quilt is a small flag bearing the emblem of the Albacore Club, the yacht club owned by Noah Cross. Jake is confronted by the host who has figured out Jake's ruse. The man takes him out to the lobby where Mulvihill is waiting. Jake tells Evelyn to bring the car around and then severely beats Mulvihill and barely escapes when the short thug who slashed his nose shows up.
Back at Evelyn's house, Gittes and Evelyn share a romantic interlude. As they lie on the bed afterward, Evelyn asks Jake about his past as a cop. He tells her he worked in Chinatown and was responsible for a woman "being hurt", possibly killed because of his actions. The phone rings and Evelyn has a cryptic conversation with someone, then informs Jake that she has to leave for a little while. She gravely asks him to trust her and wait for her to return.
Gittes takes Mulwray's car and follows Evelyn to a middle-class house and sees Mulwray's girlfriend crying. He confronts Evelyn as she leaves, who claims the young woman is her sister, who was distraught because she had just learned about Mulwray's death in the newspaper. Later that night, Jake receives a call at home from a detective named Loach, Escobar's partner, telling him to meet him at a specific address. When he gets there he finds that Ida Sessions is murdered and Escobar and Loach are waiting for him. When Jake asks how they knew to call him Escobar shows Jake his phone number written near the phone. Escobar also points out that he knows the coroner's report proves that salt water was found in Mulwray's lungs even though the body was found in a freshwater reservoir, a fact that Jake had discovered earlier but withheld. He demands that Jake turn over any incriminating photos that may reveal Mulwray's murderer's identity. Escobar's chief suspect is Evelyn herself.
Under pressure from Escobar threatening to revoke his PI's license, Jake returns to Evelyn's mansion looking for her. Evelyn's Japanese gardener is working in the backyard and drops a minor comment about "salt water being bad for the *grass*". The man's accent had masked over the actual word he was using to describe the problem when he'd said "bad for the glass." Jake has the man fish out the shiny object he'd noticed in the pool before: it's a pair of eyeglasses.
Gittes confronts Evelyn at the small house where she'd been keeping the young girl. Evelyn reveals that the blond girl, Katherine, is both her sister and her daughter, born from an incestuous relationship she had with her father years before. Gittes asks Evelyn if her father raped her and she shakes her head no. It remains unclear whether the act was consensual or not. It is apparent also that Evelyn resents her father for taking advantage of her in a relationship considered unnatural. Gittes then chooses to help Evelyn escape. Evelyn also states that the eyeglasses Jake found in her back yard pond could not have been her husband's because they are bifocals. Gittes arranges for the two women to flee to Mexico on a fishing boat owned by another of Jake's clients and instructs Evelyn to meet him at her butler's address in Chinatown.
Evelyn leaves, and Cross arrives with Mulvihill under the pretext that Gittes has found the girl; however, Gittes confronts Cross with the accusation of murder and the glasses. Cross had Mulwray drowned in the saltwater pond at his own house and lost his own glasses in the pond during the act. Jake asks Cross about the water scandal; Cross blithely tells him that he plans to create a community in the desert with an abundant fresh water supply. The real estate revenues from the sales of the land will generate many millions of dollars for him. Cross considers the plan a way of buying the future, essentially insuring that his family will reap the benefits from such a deal for many years. When Jake pointedly asks Cross about the relationship with his daughter, Cross confidently says "Most people never have to face the fact that, at the right time and the right place, they're capable of anything". Cross then orders Mulvihill to seize the glasses, the only physical evidence Jake has and forces Gittes to take him to Evelyn's butler's address in Chinatown. When Gittes arrives at Evelyn's hiding place in Chinatown, the police are already there and arrest Gittes on conspiracy and withholding evidence. Jake vainly tries to explain Cross' plan to Escobar, who won't listen.
Evelyn appears with her daughter, trying to drive away in her car. When Cross approaches Katherine, demanding custody of her, Evelyn pushes him back, shoots him in the arm with a small pistol and starts her car. As Evelyn is driving away, the police open fire and Evelyn is shot and killed. Cross clutches the hysterical Katherine, taking her out of the car, as a devastated Gittes is comforted by his associates, who urge him to walk away: "Forget it, Jake. It's Chinatown."
The plot is based in part on real events that formed the California Water Wars, in which William Mulholland acted on behalf of Los Angeles interests to secure water rights in the Owens Valley.