Andrés Ximénez built the house now bearing his name in St. Augustine in 1798, during Florida's second period of Spanish rule. Constructed of coquina with substantial rooms and commercial space, the building combined residence, store, warehouse, and tavern functions near the colonial town's waterfront economy. Ximénez, his family, enslaved workers, employees, customers, sailors, and travelers participated in a port community shaped by Spanish law, Atlantic trade, military occupation, and exchange with nearby plantations and Indigenous peoples.

Florida passed from Spain to the United States in 1821, and St. Augustine's economy adjusted to new laws, migration, and American territorial government. Margaret Cook purchased the property in the 1830s and operated it as a boardinghouse. Her business marked the beginning of a long period in which women managed the building and provided paid lodging, meals, washing, and social space. Guest rooms and service areas reveal both the independence available to some proprietors and the demanding labor performed by servants.

Sarah Petty Anderson later ran the establishment, and Louisa Fatio acquired it in 1855. Fatio belonged to a family long associated with Florida and continued the boardinghouse through years of sectional crisis and war. Guests arrived for health, business, government, and tourism, while free and enslaved African Americans performed much of the domestic work before emancipation. The Civil War disrupted travel and commerce, but St. Augustine remained occupied by United States forces for most of the conflict, sparing the building from battlefield destruction.

After Fatio's death, the property experienced new owners and uses as St. Augustine became a major tourist destination. Railroads, hotels, and promotion of the city's Spanish colonial image transformed the local economy. Preservationists increasingly valued the Ximénez-Fatio House as evidence of life before the resort era. The National Society of the Colonial Dames of America in Florida acquired the property in 1939, undertook research and restoration, and developed rooms interpreting successive boardinghouse proprietors and their guests.

The museum now presents architecture, furnishings, archaeology, commerce, and women's enterprise across Spanish and American St. Augustine. Interpretation has broadened beyond genteel visitors to include enslaved and free laborers, servants, cooks, laundresses, and craftspeople who sustained the household. Conservation must distinguish original coquina and surviving features from later restoration choices. The site's particular value lies in continuity of work: a merchant's mixed-use property became a female-run boardinghouse and then a museum where domestic labor, migration, tourism, and changing sovereignty can be examined within the same rooms.