Perhaps the worst accusation made against the Know Nothing party, in the mind of the average Southerner, native born or not, was that the Know Nothing party secretly supported abolitionism. As a secret organization, with secret handshakes and the like, it was easy for outsiders to view Know Nothings with suspicion. In the South, abolitionists were seen as evangelical extremists. For the average Irish Catholic, extreme Protestantism awoke their greatest fears. The accusation of supporting abolition of slavery hurt the Know Nothings more than the anti-Irish prejudice.
Father Patrick Lynch, also a slave holder, wrote an article entitled, “The Secret Sect.” He argued that the blatant Americanism so prevalent in the North was wedded to abolitionism. By “Americanism,” he meant the anti-immigrant fervor. He argued that Irish Catholics were loyal to the South and its institutions, while the American Party was not. Fr. Lynch lived in Charleston, and was one of the first native born priests in America. Abolitionists were indeed quite evangelical. They were often seen as fanatics. In truth, the Know Nothings tried to remain neutral on slavery. In a time when the slavery divide was increasingly pronounced, neutrality itself raised suspicion. The nascent Republican party became more attractive. Many Northern Know Nothings joined the Republican party by 1860. Southern Know Nothings were then left with a party seen as un-patriotic. The Democrat party gained even more members.
And, in the process, the Irish became more secure in their Irishness.
David T. Gleeson, The Irish in the South, 1815-1877 (Chapel Hill, N.C.: Univ. of North Carolina Press 1995), p. 79-80, 119-120.
Irish Confederates 