Martha Jefferson, usually called Patsy by her family, was born at Monticello in Albemarle County, Virginia, on September 27, 1772. She was the first child of Thomas Jefferson and Martha Wayles Skelton Jefferson and one of only two of their six children to survive into adulthood. Her earliest years passed within the plantation household at Monticello, where her parents provided her first instruction. The comfort and learning available to the Jefferson family depended upon the labor of enslaved people, including members of the Hemings family whom the Jeffersons acquired through the Wayles inheritance.

She watched her parents grieve the deaths of several infant children. When Thomas Jefferson became governor of Virginia in 1779, the household moved first to Williamsburg and then to Richmond. British military operations forced the family to flee Richmond in 1781. The following year, when Patsy was nearly ten, her mother died after childbirth. Patsy later remembered accompanying her devastated father on long rides and described herself as the constant companion who witnessed his grief.

In 1784, Jefferson took Patsy with him when he traveled to France as an American diplomat. Paris exposed the young Virginian to languages, manners, music, art, and political society on a scale unavailable at Monticello. Jefferson placed her at the Abbaye Royale de Panthemont, a prestigious convent school attended by daughters of the French elite. Although raised in an Anglican household, she became sufficiently attracted to Catholic religious life that her interest alarmed her father, who removed her and her younger sister Maria from the convent. Patsy nevertheless received an unusually broad education and became fluent or capable in several languages.

She returned to Virginia with Jefferson in 1789. On February 23, 1790, at seventeen, she married her third cousin Thomas Mann Randolph Jr.