Human-AI

We Tested Our Live AI Presentation With 150 Users. Then We Took It to a Hotel.

By Keith Mangold · 2026-09-01

Once we finally found the right presentation, I did the completely reasonable thing and turned a seven-minute conference segment into a software project.

The concept was called Fix It. Automate It. Or Kill It. Audience members would scan a QR code and describe a recurring process that made people in their organization groan. Our AI presenter, Jarvi, would organize the responses into common patterns. I would choose one of those patterns, and everyone would vote on their phones before seeing Jarvi’s recommendation.

The point was not to present Jarvi as the smartest thing in the room. AI is useful for sorting messy language, finding repeated themes, and proposing a starting point. It does not know the entire organization, understand every constraint, or accept responsibility for what happens next.

Jarvi would find the pattern. The room would challenge the recommendation. A human would still decide.

Building the experience

We built a custom web application with separate experiences for the presenter and the audience. The presenter controlled when submissions opened, when Jarvi analyzed them, when voting began, and when each result appeared. The software was not allowed to advance the presentation simply because an AI response had finished.

Jarvi also got a voice and a personality. The goal was to make the experience feel like a live collaboration rather than a standard presentation with a polling feature bolted onto it. Jarvi could react to the room, introduce the voting rounds, reveal its recommendations, and participate in the discussion without pretending to be an omniscient answer machine.

The planned experience included two audience questions. After those rounds, Jarvi would turn the framework on itself and ask the audience whether its own performance should be fixed, automated, or killed. It was a self-review, a final joke, and another way to reinforce that AI should not be exempt from scrutiny just because it is AI.

We thought carefully about what could go wrong. The application had presenter controls, retries, fallback results, refresh recovery, seeded demo data, and fixed summary slides that did not require another AI request. We tested the participant flow, the voting flow, and the presenter’s ability to take back control.

We also ran a production load test with 150 simulated participants. All 150 checked in and submitted ideas. The system handled more than a thousand requests without losing one. The response times were good, the database stayed stable, and the application behaved the way we expected.

That test proved the system could handle the planned audience volume under controlled conditions. It did not prove what would happen when we put a ballroom full of real phones on conference hotel Wi-Fi.

What happened during the presentation

Some audience members encountered random errors when they tried to vote. They would tap a button, receive an error, and then try again. Eventually, the vote would usually go through.

Hotel Wi-Fi is the obvious explanation, and it may be the correct one. I cannot honestly claim that without reviewing evidence we did not capture in the moment. It could have been the network, a delayed server request, a browser issue, something in our application, or a combination of several things.

What I know is that people occasionally had to retry, which is not the experience we designed or saw during testing.

Then Jarvi stopped speaking new text.

It did not fail in a spectacular way that made the problem obvious. It would reach the point where it needed to say something new and then sit there. The presentation stopped advancing while Jarvi waited for a response that never arrived.

Because of that, we never reached the second audience question. We also did not get to the final round where the audience would review Jarvi’s performance. I took over and spoke through the summary and final slides myself.

The ending was fine because I knew the material and still had control of the presentation. It nevertheless sucked that the complete experience we had designed, built, and tested did not work from beginning to end.

I do not want to rewrite the outcome as a flawless success simply because people liked the idea. The system did not work end to end in the room. That is the operational truth.

The presentation still connected

Afterward, I received messages from people saying how neat it was to see something like that at a conference and that they appreciated being invited to experiment with me.

People participated even when the technology added friction. They retried their votes. They watched us use their responses instead of talking through canned examples. When Jarvi stopped moving the experience forward, I took back control and we finished the conversation together.

The technical problems did not erase the value of the idea. If anything, they reinforced one of its central points: AI is part of a process, not a substitute for understanding and owning the process.

A polished demo can impress people, but an honest experiment can make them feel like they were part of something. I would still prefer the honest experiment to work correctly.

What I would change next time

The biggest change would be separating the presentation system from the public participation system. I would run the full presenter experience locally on my laptop, including every required slide, transition, and spoken line. The audience could still submit responses and vote through the public website, but a failed public request would not be able to stall the presentation.

I would also pre-generate or cache anything Jarvi absolutely needs to say. Live-generated speech is interesting until the request hangs and your AI presenter develops stage fright. Anything that must be generated live should have a firm time limit. If Jarvi does not respond within a few seconds, control should return to me automatically.

The human takeover should be part of the primary design rather than treated as an emergency procedure. We always knew I might need to step in, but the transition should have been faster and more natural. A live AI presentation should assume that the AI may occasionally fail and make the handoff feel intentional.

Finally, I would treat venue testing as its own category. A 150-participant simulation tests capacity. It does not test a hotel network, a ballroom full of different phones, or the complete chain of services required to generate and speak a live response. Those are different risks and need different tests.

The experience taught me that designing the happy path is not the same as designing the whole system. We had thought about failure, but too many parts of the presentation still depended on the same public chain working correctly at the same time.

The lesson is not that AI is amazing, and it is not that AI failed. The lesson is that someone still has to understand the entire process, decide where it can break, and take responsibility when it does.

Architecture comes before automation. You do not pour the concrete and then decide where the doors go. Apparently, you also should not trust the hotel Wi-Fi with every door in the building.