Strictly speaking, an epistolary novel is a novel whose story is told through a series of letters. Benét's Reader's Encyclopedia explains that the form was first popularized by the 18th century novels Pamela and Clarissa Harlowe by Samuel Richardson. Some definitions of the form stretch to include diary entries and other documents.
In a novel otherwise told in third person, letters allow the reader to hear the characters' voices more intimately. They also give an impression of immediacy and authenticity. Unlike works of the 18th century, contemporary novels rarely rely solely on letters to tell a story.
Examples: Although A.S. Byatt's Possession is not an entirely epistolary novel, much of the story is told through letters.