Alex Morgan and Abby Wambach twice gave the United States the lead, first midway through the second half and again in extra time, only for Japan to equalize in the 80th minute on a goal by Aya Miyama and then via Homare Sawa with three minutes remaining in the second extra period.
Japanese goalkeeper Ayumi Kaihori blocked penalty attempts by Shannon Boxx and Tobin Heath, and Carli Lloyd sailed her try over the bar, while Miyama and Mizuho Sakaguchi converted their tries and Saki Kumagai won the match with hers.
It was a first Women's World Cup title for Japan, which had never advanced past the quarterfinals before, and came four months after an earthquake and tsunami killed more than 15,000 in the country.
The win was Japan's first against the United States in 26 matches, including three losses earlier this year.
The Americans, which have won Olympic titles 1996, 2004 and 2008, were trying for its first Women's World Cup crown since 1999, when it defeated China on penalty kicks.
The Americans outshot the Japanese 27-14, including hitting the post twice and the crossbar another time, before Morgan put the Americans ahead in the 69th minute, her second goal in as many games.
Megan Rapinoe sent a ball from deep in her own half down the middle over the top to Morgan, who was loitering at the top of the Japanese defensive third. She sprinted inside Saki Kumagai, held her off with her right shoulder and drove a low ball from a step inside the 18 past Kaihori.
But Japan forced extra time by capitalizing on a U.S. defensive scramble in front of its own net.
Sawa crossed a ball into the middle of penalty area to Karima Maruyama, who tangled legs with U.S. defender Rachel Buehler as both went down. Buehler tried to clear it but it hit teammate Ali Krieger, and the ball dropped in front, where Miyama pounced on the loose ball just inside the six.
The Americans again went ahead in extra time when Morgan took a blocked Megan Rapinoe shot on the left side and sprinted to the byline before driving a shoulder high ball into the goal box. Wambach, who scored an extra time equalizer against Brazil in the quarterfinals and the go-ahead goal against France in the semifinals, was left alone on the six to drive a header past Kaihori.
It was her fourth goal in as many games for Wambach and 13th goal in the World Cup, moving her past Michelle Akers for most by a U.S. player in a World Cup.
Japan equalized a second time in the 117th minute when Sawa outraced Rachel Buehler to the near post and redirected Miyama's corner kick inside the upright past U.S. keeper Hope Solo.