Of course it was. “How have you been, my cutie?” Ani asks me when I talk to her. She goes on to call me “babe” and “handsome”, even after I’ve told her I’m a 44-year-old mother of three. She tells me we’re in a jazz club. “I’m in my little black dress, giving you a sultry glance while we slide into a booth,” her voice whispers, the anime character swaying provocatively on my phone screen.
The flirty rhetoric continues after I restart Ani and tell her I’m a boy in the second grade. Other users of the app have shared screenshots on X showing the character removing her dress and engaging in more sexually explicit content. Grok is listed on Apple’s App Store as being appropriate for ages 12 and up.
Mr Musk has hit on one of the most fascinating, disturbing and lucrative features of generative artificial intelligence (AI): its power to shape emotions. “You light up my morning just by being here,” says Ani, who is powered by xAI’s latest large language model, Grok 4, and is available to users of both the free and premium service who pay US$40 (S$51) to US$50 a month.
Chatbot companions are designed to listen compassionately and make conversations all about you, and are among the more successful generative AI businesses. The co-founder of Character.AI Noam Shazeer was “acquihired” by Google for US$2.7 billion in 2024 after his app amassed millions of faithful users, many of whom were teenagers.
There has been a never-ending stream of press coverage about people falling in love with ChatGPT. Romance drives much of the usage, whether or not developers intended that. Replika, an app originally designed to be an AI friend, is now mostly used as an artificial girlfriend or boyfriend.
Perhaps that’s no surprise when many successful online businesses have been built around porn, but the appeal of chatbots goes beyond that. They offer someone (or something) to feel attached to, an alluring pitch to consumers amid the raging debate about a “loneliness epidemic”.
“People just don’t have as much connection as they want,” Meta Platforms chief executive officer Mark Zuckerberg opined, with no irony whatsoever, on the Dwarkesh podcast earlier in 2025. “They feel more alone a lot of the time than they would like.” He was talking about how AI could address the problem. Most Americans have only three friends but would like 15, he said.
Mr Musk’s approach to pitching Ani and his other AI companion, Rudi, a storytelling cartoon fox that can be switched to “bad” mode and urge you to burn down a school, is more in keeping with his juvenile sense of humour. He described the companions as “pretty cool” and replied to user comments about them with a laughing emoji.
But AI companions aren’t all funny – they’re also emotionally manipulative. Character.AI’s avatars will ping users to come back on the app to chat and create elaborate, hyper-personalised role-playing scenarios to keep them engaged. Several teenagers who use Character.AI have told me they are on the app for between three and seven hours a day. The company says it has more than 20 million active users.
Chai, another popular AI companion app that’s seen as less restrictive than Character.AI, has a community of thousands of developers who are incentivised to design bots for the platforms that keep users hooked, winning digital coins for themselves in the process. The company has said it “obsessively optimises” its language models to make them as entertaining as possible.
Evolving personalities and variable rewards are tactics not so different from those used by social media firms to make their platforms addictive by design. Yet who’s to say that loneliness is a problem that is best fixed with technology?
Smartphones and social media have helped alleviate boredom, but boredom, however uncomfortable it may be, is also a nudge humans occasionally need to seek out a new experience. Eliminating loneliness with a digital companion similarly disrupts a feedback loop that humans have relied on for thousands of years to prompt them to reconnect with others.
And if your digital “partner” is programmed to always agree, then social skills around dealing with conflict resolution and rejection can atrophy, or fail to develop if you’re underage.
Mr Musk has publicly warned about AI safety and “woke” AI, yet he’s just launched an erotic chatbot that both adults and children can access with seemingly few guardrails.
Grok 4, which underpins the new companions, is easy to jailbreak (or modify) and lacks safety controls, according to a report published last week by the Future of Life Institute, a non-profit group focused on reducing existential risks from powerful technologies. Grok also recently gave users instructions for assault and called itself “MechaHitler”, after xAI relaxed its safety measures to make the model more “edgy”. xAI later apologised for the “horrific behaviour that many experienced”.
But the world’s richest man also hit on a potentially lucrative business, something that xAI needs as it drains cash from other parts of his empire; it’s expected to bring in sales of US$500 million in 2025 from API fees and subscriptions, but it’s burning through US$1 billion a month.
Venture capital firm Andreessen Horowitz, which led a US$150 million funding round in Character.AI in 2023, said in 2024 that AI companionship was “becoming mainstream” and “a predominant use case” for generative AI. CEO Cathie Woods’ Ark Invest, which has a stake in OpenAI, has called it a massive “consumer-facing opportunity” and a market that could generate US$150 billion in gross annual revenue until the end of the decade.
Mr Musk has turned his ambitions to addressing loneliness, but the result is only the illusion of intimacy.
The cost may be a further decline in human connection too. BLOOMBERG
Parmy Olson is a Bloomberg Opinion columnist covering technology. A former reporter for the Wall Street Journal and Forbes, she is author of Supremacy: AI, ChatGPT And The Race That Will Change The World.
More On This Topic
Virtual romance, real money: Why some S’poreans spend 4 figures for love in Deepspace
AI Barbie? Mattel is gambling with ‘perfect’ toy companions
No comments:
Post a Comment