Finding it hard to make ends meet can impoverish the brain and reduce your ability to think, say scientists.
Financial worries tax the brain so much they create a "cognitive deficit" equivalent to a 13-point loss in IQ, a study found.
The problem is distinct from the effects of stress and results from too much "mental bandwidth" being used to fret about money.
Someone overwhelmed with worries about rent, feeding and clothing children, and paying household bills can suffer a genuine mental handicap, the research shows.
This in turn may lead to poor decisions, such as racking up debt, creating even more difficulties in a vicious cycle.
Economics professor Sendhil Mullainathan, from Harvard University in the US, said: "Our results suggest that when you're poor, money is not the only thing in short supply. Cognitive capacity is also stretched thin.
"That's not to say that poor people are less intelligent than others. What we show is that the same person experiencing poverty suffers a cognitive deficit, as opposed to when they're not experiencing poverty.
"What happens is that your effective capacity gets smaller because you have all these other things on your mind. You have less mind to give to everything else.
"Imagine you're sitting in front of a computer and it's just incredibly slow. But then you realise that it's working in the background to play a huge video that's downloading. It's not that the computer is slow, it's that it's doing something else, so it seems slow to you. I think that's the heart of what we're trying to say."
Prof Mullanathan's team carried out a series of experiments in the US and India to highlight the mental cost of poverty.
In the first, conducted in a shopping mall in New Jersey, around 400 people were randomly chosen and asked to ponder how they would solve hypothetical financial problems, such as paying for a car repair.
Some problems were easier, that is, cheaper, to sort out than others. For instance, the car repair bill could be either 150 dollars, or 1,500.
At the same time, the volunteers had to undergo simple computer tests of IQ and mental performance.
They were split into "poor" and "rich" groups based on their income which ranged from 20,000 dollars per year to around 70,000 dollars.
The results, reported in the journal Science, show that when the financial problems were not too severe, both groups performed equally well in the tests. But when they were forced to consider difficult, costly problems, people with lower incomes had significantly worse scores.
In fact, the effect was so strong that for those generally preoccupied with money, merely thinking about a tricky financial problem led to a 13-point dip in IQ. That is on the same scale as losing an entire night's sleep.
The scientists followed up their study with a visit to rural India where they tested a group of sugar cane farmers who rely on the annual harvest for at least 60% of their income.
Since the harvests occur once a year, the farmers find themselves poor before a harvest and rich after it. Given the same tests as the New Jersey shoppers, they did significantly better when they sat them after the harvest.
The impact of poverty on mental capacity reflects a more general phenomenon related to scarcity, say the researchers.
Lacking something, whether it be money, time, social ties or even calories, puts a strain on the brain.
Co-author Jiaying Zhao, from Princeton University in the US, said: "These findings fit in with our story of how scarcity captures attention. It consumes your mental bandwidth.
"Just asking a poor person to think about hypothetical financial problems reduces mental bandwidth. This is an acute, immediate impact and has implications for scarcity of resources of any kind."
Professor Eldar Shafir, another member of the Princeton team, said: "When you're poor you can't say, 'I've had enough, I'm not going to be poor any more, or 'forget it, I just won't give my kids dinner, or pay rent this month'.
"Poverty imposes a much stronger load that's not optional and in very many cases is long lasting. It's not a choice you're making, you're just reduced to a few options. This is not something you see with many other types of scarcity."
Services to the poor should take account of the mental effect of poverty, for instance by providing simpler forms and making it easier to seek assistance, said Prof Shafir.
"The poor, who our research suggests are bound to make more mistakes and pay more dearly for errors, inhabit contexts often not designed to help."