A UK-based study quizzed 200 students on their caffeine intake and found those with the highest consumption were also more prone to report seeing, or hearing, things that were not there.
Those who consumed a daily equivalent of seven cups of instant coffee or more - high caffeine users - were three times more likely to have extra-sensory experiences than low users, who had less than one cup daily.
The Durham University study took in all caffeine consumption including coffee but also tea, caffeinated energy drinks or chocolate bars and caffeine pills.
"This is a first step towards looking at the wider factors associated with hallucinations," said lead author, Simon Jones, a PhD student at the university's psychology department.
"Previous research has highlighted a number of important factors, such as childhood trauma, which may lead to clinically relevant hallucinations.
"Given the link between food and mood, and particularly between caffeine and the body's response to stress, it seems sensible to examine what a nutritional perspective may add."
When under stress, the body releases a stress hormone called cortisol. More of this stress hormone is released in response to stress when people have recently had caffeine.
It is this extra boost of cortisol which may link caffeine intake with an increased tendency to hallucinate, say the scientists.
"However, one interpretation may be that those students who were more prone to hallucinations used caffeine to help cope with their experiences," said study co-author Dr Charles Fernyhough.
"More work is needed to establish whether caffeine consumption, and nutrition in general, has an impact on those kinds of hallucination that cause distress."
People taking part in the study reported "seeing things that were not there, hearing voices, and sensing the presence of dead people".
Mr Jones said such hallucinations were not necessarily a sign of mental illness, and around three per cent of people regularly heard such voices.
Results of the study are published in the academic journal Personality and Individual Differences.