The property that became Varner-Hogg Plantation began functioning in 1824 as an agricultural operation established by Martin Varner, one of Stephen F. Austin's original colonists, who used enslaved labor on a small scale to farm the land and operate a rum distillery on his roughly 4,400-acre grant. In 1834 Columbus R. Patton of Kentucky purchased the land and transformed it into a large commercial sugar and cotton plantation, forcing between 40 and over 100 enslaved people to clear fields, construct brick buildings and infrastructure, and produce sugar, molasses, and cotton for market; by the late 1840s the plantation was a significant regional producer. After the Patton family's control ended in the 1850s, the land passed through several owners and shifted from sugar to cotton and eventually cattle ranching, employing Black cowboys in the postbellum period. In 1901 Governor James Stephen Hogg purchased the property primarily for its oil potential, and it remained a working ranch and family retreat under Hogg ownership; his daughter Ima Hogg later furnished the main house with antiques and decorative arts reflecting her father's interests before donating the site to the State of Texas in 1958. Today Varner-Hogg Plantation functions as a Texas Historical Commission historic house museum and state historic site, interpreting both its origins as a slave-labor plantation and its twentieth-century life as an oil-era Hogg family property, through guided tours of the house and grounds.