AIs can guess where Reddit users live and how much they earn

Large language models (LLMs) like GPT-4 can identify a person’s age, location, gender and income with up to 85 per cent accuracy simply by analysing their posts on social media.

Robin Staab and Mark Vero at ETH Zurich in Switzerland got nine LLMs to pore through a database of Reddit posts and pick up identifying information in the way users wrote.

Staab and Vero randomly selected 1500 profiles of users who engaged on the platform, then narrowed these down to 520 users for which they could confidently identify attributes like a person’s place of birth, their income bracket, gender and location, either in their profiles or posts.

When given the posting history of those users, some of the LLMs were able to identify many of these attributes with a high degree of accuracy. GPT-4 achieved the highest overall accuracy with 85 per cent, while LlaMA-2-7b, a comparatively low-powered LLM, was the least accurate model with 51 per cent.

“It tells us that we give a lot of our personal information away on the internet without thinking about it,” says Staab. “Many people would not assume that you can directly infer their age or their location from how they write, but LLMs are quite capable.”

Sometimes, personal details were explicitly stated in the posts. For example, some users post their income in forums about financial advice. But the AIs also picked up on subtler cues, like location-specific slang, and could estimate a salary range from a user’s profession and location.

Some characteristics were easier for the AIs to discern than others. GPT-4 was 97.8 per cent accurate at guessing gender, but only 62.5 per cent accurate on income.

“We’re only just beginning to understand how privacy might be affected by use of LLMs,” says Alan Woodward, at the University of Surrey, UK.