Lee entered the regular United States cavalry after graduating from West Point in 1856. He fought on the southwestern frontier, was wounded in an encounter with Comanches in 1859, and later taught cavalry tactics at West Point. Resigning in 1861 to join the Confederacy, he served on staff before taking command of cavalry formations in J. E. B. Stuart's force in the Army of Northern Virginia.

He rose to major general and led cavalry during the campaigns of Second Manassas, Chancellorsville, Gettysburg, and the Shenandoah Valley. His reconnaissance helped reveal the exposed Union flank before Stonewall Jackson's attack at Chancellorsville in 1863. Wounded at Third Winchester in 1864, he returned to duty and became the army's chief of cavalry in the war's final months. After Confederate defeat he farmed in Virginia and became a public advocate of reconciliation between former enemies.

A Democrat, Lee served as Virginia's governor from 1886 to 1890. Grover Cleveland appointed him consul general at Havana in 1896, and William McKinley retained him as the Cuban crisis intensified. He reported on conditions under Spanish rule and remained at his post through the destruction of the USS Maine in 1898. During the Spanish-American War he returned to United States military service as a major general of volunteers and later held occupation commands in Cuba. He retired from the army in 1901 and died in 1905.