A friend was telling me the other day about why Google snapped up Character AI. Apparently, the big deal wasn’t just the cool AI girlfriends everyone were chatting to, but that they’d cracked the code on running these things without costing a fortune. Like, doing the AI magic – the “inference” part – at a scale that wouldn’t bankrupt you.

The adoption curve of any new tech usually starts with porn and parasocial relationships. Look at any list of GenAI products with the highest MAU - most of them are about curing loneliness. AI girlfriends, AI companions, AI therapists. But running these services is expensive. Like really expensive. The inference costs can kill you before you even figure out product-market fit. This is what made Character AI special. They could serve millions of users without bleeding money on each conversation. And Google probably saw this and went “hey, this is exactly what we need for Gemini”. Now that massive amount of money they paid for Noam Shazeer has started to look like a pretty good investment.

Think about it. Google is now giving away these flash models for free on AI Studio. You can’t do that unless you’ve figured out how to make inference really cheap. Cheap enough that you can offer it as a free product without destroying your unit economics.

Pre-training has hit diminishing returns. Everyone knows this. That’s why you’re seeing this shift towards chain-of-thought prompting and inference-time optimizations. It’s probably why Anthropic is working on improving Sonnet instead of launching something bigger. And OpenAI is launching O3 and not GPT 5.