Lee entered the United States Navy as a midshipman in December 1820 and became a lieutenant in 1828. His early sea service developed into a career of shipboard and shore commands. During the Mexican War he participated in the operations at Veracruz. Promoted to commander in 1850, he commanded the steam frigate Mississippi during Commodore Matthew Perry's expedition to East Asia, including the visit to Japan in 1853.

His peacetime assignments also included duties at the Naval Academy and command at the Philadelphia Navy Yard. Virginia's secession in April 1861 brought a break with the service in which he had spent four decades. He offered his resignation, which the federal authorities did not accept, and was dismissed before joining the Confederate Navy as a commander. His family's ties to Virginia influenced that choice, though his private comments later expressed regret about the secession crisis.

Lee commanded the Gosport Navy Yard after its abandonment by United States forces and worked to maintain the Confederate naval establishment there. When Union forces recovered Norfolk in 1862, he moved to the defenses at Drewry's Bluff on the James River. In May 1864 he became chief of the Bureau of Orders and Detail, administering officer assignments and personnel matters for the Confederate Navy. Promoted to captain, he remained in that work until the Confederacy collapsed. After the war he farmed at Richland in Stafford County, Virginia, where he died in July 1869.