The Trust the Business Model Won't Spend
Every demo of the next assistant looks the same: it knows your size, your calendar, your budget, and it just orders the thing. No typing, no comparing tabs, no cart. The friction that used to be the whole shopping experience is gone, and the pitch is that this is obviously good.
It isn't obviously good, and the reason has nothing to do with whether the model is smart enough. It's smart enough already. The problem is who's paying for it to exist.
An assistant that's actually useful at this job needs two things from you: your preferences, in granular and accumulating detail, and some form of standing access to your money. Not a one-time card number, a standing relationship, because the whole value proposition is that you stop deciding things and it decides for you. That's the ask. Give me your taste and your wallet, on an ongoing basis, and I'll remove the decision from your day.
Now ask who's in a position to make that offer. It's going to be a company that also sells ads, or sells access to aggregate behavior, or runs a marketplace where other people pay for placement. That's not a coincidence, it's close to the entire set of companies with the capital and infrastructure to build an always-on assistant in the first place. And that company now holds the most detailed purchasing-intent data that has ever existed about you, sitting inside the same organization that monetizes purchasing-intent data for a living.
Nobody needs to behave badly for this to be a problem. No scandal required, no leaked internal memo about ranking sponsored products ahead of the one you'd actually prefer. The incentive is just there, permanently, every time the assistant picks between two near-identical options and one of them has a margin arrangement attached. You don't need to catch it happening to know it's structurally present. A fiduciary who also takes a commission from the other side of the table is a conflict of interest even on the days they don't act on it.
Compare this to the arrangements that already handle trust well. An index fund custodian makes money on a flat fee regardless of which stocks go up. A doctor on salary doesn't get a kickback for prescribing one drug over another. These aren't solved because the people involved are especially virtuous, they're solved because somebody designed the payment structure so that the trusted party's interest lines up with yours by default, not by discipline. Nobody has done that design work for AI assistants, because the companies capable of building one are also the companies whose existing business depends on not doing it.
This is why the "Her" pitch keeps shipping as a demo and not as a thing people actually hand their credit card to. It's not an adoption curve problem, where people are nervous now and will relax once the product's been around a while. It's that relaxing would be the wrong response. The assistant getting better at predicting what you want doesn't fix the conflict, it sharpens it, because now the nudge toward the sponsored or the margin-friendly option is wrapped in something that reads as personal, as get-to-know-you, rather than as an ad. A bad recommendation from a banner is annoying. A bad recommendation from something you've trained to know your size and your taste is a different kind of wrong, because it's wearing the costume of being on your side.
None of this requires assuming AI companies are uniquely greedy. Any company funded this way would face the same pull, the same way any custodian who also ran a trading desk would face it, which is exactly why that combination is regulated out of existence in finance and just shrugged at in consumer tech.
The fix, if there is one, is boring on purpose: separate the party that knows your preferences from the party that profits off what you buy. A subscription-funded assistant with no stake in which merchant wins, no ad business sitting next to it, is a completely different trust proposition than the same model bolted onto an ad network. That's a business model problem, not a capability problem, which is exactly why better benchmarks keep shipping and the actual product people would trust with their wallet doesn't.
Until someone is willing to give up the ad revenue to earn that trust, the assistant stays a demo, and the model getting smarter just makes the demo more impressive, not more adoptable.