Nah, forget it. You are welcome here, at least on my part. Stay and have a chat about whatever you like.
I'd say he expressed his contempt towards them, rather than insulting. If there was no such idea, everybody would be an atheist, thus eliminating the need for a special noun to refer to...