It's not tuned to you
Many of us have gone looking for an email we knew had arrived, and found it two days later in Promotions.
Inbox sorting has been automated for years. Gmail's Priority Inbox, its Promotions and Updates tabs, and Outlook's Focused inbox, all real machine learning, not the old keyword rules. It's already being done for us. Even so, many of us still read through the whole inbox ourselves, as if it weren't.
The reason isn't that the sorter is dumb. It runs on a generic idea of what's important, and that idea isn't yours. A bill lands in Promotions, a flight change in Updates: neither is a hard call, they're just not the calls you would have made, and there's no simple way to tell it what matters to you.
That's what the distrust is made of. It's not that the machine gets it wrong sometimes; it's that it sorts by priorities that aren't yours, and you can't set your own. So you keep doing it yourself, because at least your ranking is yours.
The usual promise is that a smarter model will finally handle your inbox. A smarter model still sorts by a generic idea of important. Make it as sharp as you like, and it gets better at guessing what matters to people in general, not at knowing what matters to you. A decade of automated sorting that many of us still don't lean on is the evidence: the bottleneck isn't intelligence.
What would change it is a system tuned to your priorities, one you can adjust when it gets a call wrong. That's a design problem, not a matter of a bigger model.
The tools we already have don't fail because they're stupid. They fail because they're not tuned to you.