The statement closely mirrors what Paul says of gentile unbelievers in Romans 1:
the wrath of God is revealed from heaven against all ungodliness and unrighteousness of men, who by their unrighteousness suppress the truth. 19 For what can be known about God is plain to them, because God has shown it to them. 20 For his invisible attributes, namely, his eternal power and divine nature, have been clearly perceived, ever since the creation of the world, in the things that have been made. So they are without excuse. 21 For although they knew God, they did not honor him as God or give thanks to him, but they became futile in their thinking, and their foolish hearts were darkened.Given that the claim seems to be rather biblical, then, one may wonder why this would be a sore point of controversy even among Christians. But the controversy also touches atheists, who usually find it rather offensive to be told that they in fact believe in God, when they tell you they don't! The feeling is understandable, so what is really going on?
So the question is raised: how can one say that an atheist, who doesn't believe that God exists, in fact knows that God exists? There is a rather basic principle of epistemology (the science of how we know what we know), which says that knowledge involves at least a true belief. In fact, knowledge is much more than true belief, because one's true belief could happen to be just a lucky guess and not a case of knowledge, but whatever else knowledge requires in addition, it demands no less than true belief.
To make these two conditions plain: in a case of knowledge, the proposition needs to be true: no matter how strongly I believe that aliens landed on area 51, I cannot know this, if it is false.
And in a case of knowledge, the proposition needs to be believed: indeed, even through it is true, I cannot know that I was born in France, if I don't believe that I was.
But then it means that knowledge entails belief, and hence one who doesn't believe God exists cannot know God exists. All fair points.
To help make sense of the matter, let me take these concepts to an area of my doctoral work: the question of moral responsibility. This discussion surrounds the question of whether persons can be held morally responsible, that is praiseworthy or blameworthy for what they do, and asks about some of the circumstances which could remove moral responsibility. For example, if I freely lie to advance my career, I am presumably blameworthy for lying, but there could be circumstances in which I can be excused for speaking the same falsehoods. Coercion would be one of those: if the false confession was obtained under torture, I cannot be morally responsible for 'lying'. You get the idea.
So in this field again, lots of interesting controversies arise, but here is a condition of moral responsibility that shouldn't be too controversial, and that is of particular interest for our present question about atheists: it says that in order for a person to be morally responsible for a wrong-doing, they must know that what they are doing is wrong.
For example, if I pour poison in my wife's coffee because someone glued a 'sugar' label on the poison jar, I am not morally responsible for killing my wife, even though I most definitely killed her: I put poison in her coffee! So why am I not guilty? Because I did not know that what I was doing was wrong. One can also apply this criterion to certain cases wherein mental illness excludes moral responsibility for actions which would otherwise be culpable if they were carried out by people with normal cognitive faculties. A severely autistic person who injures their caretaker in a fit of violence can (at least in some uncontroversial cases) be excused and declared not to be morally responsible, on the grounds that they didn't know that what they were doing was wrong.
This seems reasonable enough.
But now here is the rub. Take this criterion to a different case, that of Adolf Hitler, and the question of whether he was morally responsible for organizing the death of millions of Jews and Gypsies. Was he morally guilty? I think the answer is obviously yes, but then ask our question of Hitler, now: 'did Hitler know that killing Jews and Gypsies was wrong?'
Well, in a sense, surely yes, and yet in another sense, surely no. Let me explain.
Did Hitler believe that killing Jews and Gypsies was wrong? Well, certainly not; he even believed that it was the best thing to do for the world! But if, as I established above, you can't have knowledge of a proposition without having a belief in that proposition, then it would follow that Hitler did not have the knowledge that killing Jews and Gypsies was wrong. Are we then in the painful position of having to concede: 'Hitler was not morally responsible for killing Jews and Gypsies, because he just didn't know it was wrong'? Surely that can't be right either.
So what shall we say?
I think what we must say is that our criterion for knowledge and our criterion for moral responsibility are both basically fine, but they must make room for a certain sense of knowledge in which one can know something to be the case, while (even honestly) formally professing a lack of belief in that state of affairs. Hitler knew that killing millions of Jews and Gypsies was wrong, because that moral fact is made plain to all by the light of human conscience, but he suppressed that knowledge--a morally culpable suppression, I should add; one that led him to profess a lack of belief, all the while being a suitable target of moral blame, because in a separate but very real sense, he knew even what he professed not to believe.
Applying these concepts to belief in God, we are now in a position to express the two separate senses in which Christians can coherently declare that atheists both know and don't know that God exists.
They don't know in the sense that they don't consciously hold a belief and hence lack knowledge of what they don't believe, but there is also the sense in which Christians assert that God's existence is made plain to all, and hence one who rejects it is culpably suppressing a truth which he 'knows' while (sincerely) professing not to believe it.
Now from this, two things don't follow.
First, it doesn't follow that presuppositional apologetics is correct and classical apologetics isn't. That is frankly a separate question, on which all my thoughts can be found here.
And secondly, it doesn't follow that Christians should tell atheists that they really believe in God. I personally question the wisdom of doing so in most circumstances.
But what does follow, is that Christians can coherently hold the belief that in some sense the knowledge of God is inevitable in light of general revelation, a light which Paul says is suppressed in unbelief, until the Spirit comes and shines 'the light of the knowledge of the glory of God in the face of Jesus Christ'. (2 Cor 4:6).