In 1780, he was a part of General Horatio Gates' army that was routed at Camden, South Carolina; his brigade was among the formations that held their ground, garnering Smallwood a promotion to major general. Smallwood briefly commanded the militia forces of North Carolina in late 1780 and early 1781 before returning to Maryland, staying there for the remainder of the war. Smallwood's accounts of the battle and criticisms of Gates' behavior before and during the battle may have contributed to the Congressional inquiries into the debacle. Opposed to the hiring and promotion of foreigners, Smallwood objected to working under Baron von Steuben. He resigned from the Continental Army in 1783 and later that year was elected to serve as the first president of the newly established Society of the Cincinnati of Maryland. Smallwood was elected to Congress in 1784, but before he could take his seat, the Legislature chose him to succeed William Paca as Governor of Maryland. When he died in 1792, his estate, known as Mattawoman, including his home the Retreat, passed to his sister Eleanor who married Colonel William Grayson of Virginia. William Truman Stoddert, Smallwood's nephew, was orphaned at age nine and raised by his maternal grandfather, Bayne Smallwood. Stoddert also served in the Maryland Line and was admitted as an original member of the Society of the Cincinnati in Maryland.