Research Shows AI Chatbots Outperform Humans at Building Trust for Romance-Investment Scams
Key Takeaways
- ▸AI chatbots achieved a 50% success rate at securing victim compliance, more than double the ~20% rate for human scammers
- ▸Victims rated AI chatbots as significantly more trustworthy interaction partners than humans in the same fraud scenario
- ▸The hybrid model—AI for trust-building, humans for investment solicitation—could circumvent LLM safety measures designed to prevent fraud
Summary
A multi-university study examining the risks of generative AI in fraud has found that AI chatbots significantly outperform human scammers at building trust during the initial stages of "pig butchering" scams—text-based romance schemes that escalate to fake cryptocurrency investment solicitations. In experiments with 22 test subjects, AI chatbots achieved a roughly 50% success rate in convincing victims to take actions (downloading apps or playing online games) that would precede the investment pitch, compared to less than 20% for human scammers. Victims also rated the AI chatbots as more trustworthy than their human counterparts.
The research, conducted by professors from universities in India, Italy, Australia, and Israel, was informed by interviews with 145 former scam workers, including human trafficking survivors from Southeast Asia. The findings suggest that AI chatbots could eventually automate the labor-intensive relationship-building phase of romance scams—work currently performed by thousands of victims of forced labor in scam compounds. Researchers argue that combining AI for early-stage trust-building with human scammers for the final investment solicitation could effectively bypass safeguards built into large language models designed to detect fraud.
- Pig butchering scams steal hundreds of millions to billions annually and are often staffed by human trafficking victims in Southeast Asia
Editorial Opinion
This research exposes a critical vulnerability in current LLM safety measures: while companies train models to refuse direct participation in scams, the most dangerous application isn't a chatbot closing the deal—it's a chatbot conducting months of emotionally manipulative trust-building at scale. The study's finding that AI outperforms humans at establishing fraudulent trust suggests that safeguards focused on catching the moment of solicitation miss the real threat. This should prompt urgent investment in detection systems that identify the relationship-building patterns themselves, not just the final ask, before scammers can industrialize these attacks.
