In keeping with the popular thought of the day, Finley saw the presence of blacks in America as a threat to the national well-being and the quality of life for whites. He said that free blacks were "unfavorable to our industry and morals" and that removing them would save Americans from difficulties such as interracial marriage and having to provide for poor blacks.
In December 1816, Finley traveled to Washington, D.C. There he won the immediate support of his brother-in-law, Elias B. Caldwell, Clerk of the Supreme Court, and Caldwell's friend, Francis Scott Key (author of the Star Spangled Banner), both of whom had a reputation for being friendly to Washington's free blacks. Together, the three canvassed for support, and on December 21, 1816, called an organizational meeting for the society that included some of the most powerful and influential men in the country.
In a series of meetings over the next few days, the group adopted a constitution, chose officers, and decided to call themselves the "American Society for Colonizing the Free People of Color in the United States." Bushrod Washington, the nephew of George Washington, was chosen as president of the society. Vice presidents included Finley, Henry Clay, and Richard Rush, the son of Benjamin Rush. Andrew Jackson was included on the list without his consent; in reality he was a staunch anti-colonizationist. Caldwell was the organization's secretary, and Key was on the board of managers.
The motives of the ACS members varied considerably. Some were genuine allies of free blacks, and were concerned for their welfare. Some hoped that colonization would eradicate slavery. Others wanted to maintain the institution of slavery but to rid the country of free blacks, who they believed posed a serious threat as potential fomenters of slave rebellion.
The concept of black colonization did not originate with Finley. Since 1787, efforts to find an alternative home for free blacks had sometimes been praised by blacks themselves and by staunch allies such as Anthony Benezet and Benjamin Rush. One of the most active proponents of colonization was Paul Cuffe, who felt that black people living in America would never receive the full benefits of citizenship, and that they would fare much better on the friendly shores of Africa.
The response of black Philadelphians to colonization was mixed. James Forten was a close friend of Cuffe's and a supporter of Cuffe's colonization schemes. Other prominent blacks, such as Richard Allen and Absalom Jones, knew all too well the effects of white prejudice, and African colonization seemed an attractive alternative. Yet despite their leaders' support for colonization, the common people unequivocally rejected the notion, and Philadelphia's blacks became well known as the chief opponents of the ACS.
The ACS continued its work until after the Civil War. The organization worked with the United States government to establish the African colony of Liberia, where it transported approximately 12,000 blacks over the course of its existence. Although the ACS controlled the bulk of emigration, other groups formed their own schemes. The total number of black people to emigrate from the United States to other countries was approximately 15,000.