Some notes from the Famous UUs website:
Like many others of his time (he died just one year after the founding of institutional Unitarianism in America), Jefferson was a Unitarian in theology, though not in church membership. He never joined a Unitarian congregation: there were none near his home in Virginia during his lifetime. He regularly attended Joseph Priestley's Pennsylvania church when he was nearby, and said that Priestley's theology was his own, and there is no doubt Priestley should be identified as Unitarian. Jefferson remained a member of the Episcopal congregation near his home, but removed himself from those available to become godparents, because he was not sufficiently in agreement with the trinitarian theology.
Jefferson was also a strong believer in religious freedom. He felt that there should be a "wall of seperation between Church and State" and that taxes should not be used to support religious institutions. Portions of his ideas were eventually incorporated into the 1st amendment by James Madison. The debate over the proper relationship between the government and religious organizations continues to this day. See Americans United for Separation of Church and State.
The 1st amendment to the United States Constitution: Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.
Jefferson was also instumental in the writing and introduction of the Virginia Act for Establishing Religious Freedom which was introdued when he was Governor of Virginia and was passed seven years later in 1786. When it was finally enacted, Jefferson wrote to Madison from Paris:
"It is comfortable to see the standard of reason at length erected, after so many ages during which the human mind has been held in vassalage by kings, priests and nobles: and it is honorable for us to have produced the first legislature who had the courage to declare that the reason of man may be trusted with the formation of his own opinions" (Thomas Jefferson: The Revolutionary Aristocrat, 75)
Another good example of how Jefferson belived in using inquiry and applying it to religion is in this note to his nephew, Peter Carr:
"Fix Reason firmly in her seat, and call to her tribunal every fact, every opinion. Question with boldness even the existence of a God; because, if there be one, he must more approve the homage of reason than of blindfolded fear. ... Do not be frightened from this inquiry by any fear of its consequences. If it end in a belief that there is no God, you will find incitements to virtue in the comfort and pleasantness you feel in its exercise and in the love of others which it will procure for you" (Jefferson's Works, Vol. ii., p. 217).
Thomas Jefferson died on July 4, 1826 at his Home in Monticello. The only other President that signed the Declaration of Independence (along with Jefferson) died on the same date. Supposedly, Adams' dying words were that "Thomas Jefferson survives", but Jefferson died a few hours earlier. [It's sort of complicated, but Jefferson and Adams were friends for a long period of time, until party rivalries and differences in opinion drove them apart (updated by nate). (i.e. Federalist vs. Republican) They became friends again only later in Jefferson's life.]
A good biography of Jefferson is Milton Meltzer's Thomas Jefferson: A Revolutionary Aristocrat.
To His Excellency Thomas Jefferson by Jack McLaughlin is a good annotated collected of the President's correspondence.
Jefferson Bible -- NPR's Scott speaks with Forrester Church, pastor of the All Souls Unitarian Church in New York City, about the Jefferson Bible. (6:20)