The demo always works. Someone types a question, the assistant answers in a clean paragraph, the room nods, and for a moment everyone can see the future. Then Monday comes. The team opens the same inbox, the same spreadsheet, the same WhatsApp groups, and the new thing sits in a browser tab nobody clicks.
I have watched this happen enough times to stop blaming the technology. The models are good enough for most of what a business actually needs. What breaks is everything around the model: who owns it, where it lives, what it was fed, how anyone knows it is working, and whether people trust it on a bad day.
This is a note on where AI pilots really fail, and the few decisions that keep one alive past its first week.
The demo is not the product
A demo answers the question "can this work?" A product answers a harder one: "will this still be used on a busy Tuesday, by someone who did not sit in the meeting?"
Those are different problems. The first is technical. The second is about behaviour, habit and trust, which is why it so often gets skipped. It feels soft. It is not. It is the part that decides whether the money you spent turns into hours saved or into a slide in next year's strategy deck.
So when I look at an AI project that stalled, I rarely ask what model it used. I ask five other questions.
Five places pilots break
1. Nobody owns it
A tool without an owner is a guest. It gets polite attention for a few days and then it is forgotten. Somebody on the team has to wake up responsible for it: reading what it got wrong this week, adding the missing answers, deciding what it should stop doing.
This does not need to be a technical person. In most small teams the best owner is the person who used to do the job by hand, because they know what a good answer looks like.
2. It lives outside the work
If the assistant sits in a new app, you are asking people to change where they work before they have any reason to. That is two changes at once, and people will take neither.
The pilots that survive live where the work already happens: inside the inbox, the CRM, the chat the team already uses, the form customers already fill. The less someone has to go somewhere new, the more likely they are to use it on day ten.
3. It learned the tidy version of the business
Pilots are usually built on clean examples: the ten questions everyone agrees on, the FAQ page, the brochure. Real customers do not write like the FAQ. They send voice notes, mix languages, ask three things in one message and leave out the one detail that matters.
If the assistant has only ever seen the tidy version, the first messy week teaches the team that it "does not understand our clients." That judgement is very hard to reverse. Feed it the messy reality from the start: real messages, real objections, the questions that make your team sigh.
4. Nobody decided what good looks like
"Let us see how it goes" is not a measure. Without one, the pilot is judged on mood, and mood is decided by the single most embarrassing mistake it made.
Pick one or two things you can count before you start. How many requests it handles without a person. How long customers wait for a first answer. How many hours a week the team gets back. Write the current number down, even if it is a rough estimate. Then look at it every week, not every quarter.
5. Trust was never designed
Every assistant will be wrong sometimes. What matters is what happens next. If a wrong answer goes straight to a customer with no way back, one bad day ends the project.
Design the handoff before launch. What does the assistant do when it is not sure? Who gets the conversation, how fast, and with what context, so the customer never has to repeat themselves? When people see that the system knows its limits and passes the hard cases to a human cleanly, they start trusting it with the easy ones. That is how usage grows.
What the survivors have in common
The AI projects I have seen last are almost boring in how they start:
- One job, not ten. One clear task, like answering the questions that arrive every day or preparing the follow-up after a sales call. Scope grows only after the first job is trusted.
- Inside existing tools. No new logins for the team if it can be avoided.
- A named owner who reviews it weekly and has the authority to change it.
- A designed handoff to a human, with the conversation history attached.
- One number that everyone can see move.
None of this is advanced. It is discipline. And it is the difference between a pilot and a habit.
A one-page test before you start
Before you spend anything on an AI project, try answering these on a single page:
- What is the one job it will do, in one sentence?
- Who does that job today, and will they own the assistant?
- Where will people use it, and is that somewhere they already work?
- What real, messy examples will it learn from?
- What does it do when it is unsure, and who catches it?
- What number will tell you, in four weeks, that it is working?
If you cannot answer all six, the project is not ready, and that is useful to know before the budget is gone rather than after.
Why a designer cares about this
People are sometimes surprised that a creative director spends his days on AI systems. I think the overlap is obvious. Getting a team to use an assistant is a design problem: it is about friction, habit, trust and the feeling someone has when a tool helps them instead of adding one more thing to check.
The technology is the easy part now. The work is in shaping how it fits into real days, with real people, under real pressure. That is where most pilots quietly die, and it is also where the good ones are won.
If you are about to start an AI project, or you have one sitting in a tab nobody opens, start with the one-page test. And if you would like a second pair of eyes on it, I am always happy to talk it through.
