The hardest part of building an AI travel app: when the AI is confident and wrong

TravelMe's core promise is that you describe a trip and the AI handles the rest. What I didn't fully appreciate when I started building it is how a confident AI that gives wrong answers is, in some ways, worse than an app that gives no answers at all.

This post is about the failure modes I found while testing TravelMe, how I'm handling them, and the philosophical tension at the centre of building AI travel products.

Confident and wrong

LLMs don't know when they don't know something. Ask GPT-4o about a restaurant that closed two years ago and it will recommend it enthusiastically, complete with opening hours that are no longer accurate. Ask it about a hiking trail that's currently closed due to landslide damage and it won't mention the closure, it can't, because it doesn't know.

During testing I caught TravelMe recommending:

• A museum in Lisbon that had relocated, the address was wrong by about 2km
• A beach in Sardinia described as 'quiet and local' that is now heavily developed and touristy
• A ryokan in Kyoto that had closed during COVID-19 and never reopened
• Visa requirements for Kosovo passports that were three years out of date

Each of these would have caused a real problem for a real traveller. The museum one is annoying, a wasted taxi journey. The visa one could theoretically strand someone at a border.

The disclosure problem

My first instinct was to add a disclaimer. Something like: 'AI-generated suggestions may be outdated. Always verify before booking.' But this is a band-aid, not a solution. Users don't read disclaimers. And a disclaimer doesn't fix the underlying problem, the app is presenting outdated information as confidently as current information.

The better solution, which I'm implementing now, is to be explicit in the UI about what the AI knows and what it doesn't:

• Practical information (prices, opening hours, addresses, visa requirements) is marked with a 'Verify before going' tag and linked to authoritative sources
• Experiential suggestions (what an area feels like, what type of traveller will enjoy a restaurant) are left as AI recommendations without the warning, because they're less likely to be dangerously wrong
• Time-sensitive information (weather, events, seasonal closures) is not AI-generated, it comes from live APIs or is flagged as requiring verification

Grounding the model

A technique that improved accuracy significantly was adding explicit grounding instructions to the system prompt, telling the model to acknowledge uncertainty rather than confabulate:

// Added to system prompt
`
If you are not confident about specific practical details (exact opening hours,
current prices, visa requirements, specific addresses), do not invent them.
Instead, include a note in the relevant field:
  'opening_hours_note': 'Verify current hours before visiting'

You may be confident about experiential qualities (the atmosphere of an
area, the type of cuisine, the general character of an accommodation type).
You may not be confident about facts that change over time.
`

This reduced the number of invented specifics significantly. The model became more likely to say 'typically open 9am–6pm' instead of '09:00–18:00', flagging the uncertainty through imprecise language rather than omitting the information entirely.

The philosophical tension

Here's the tension I keep coming back to: TravelMe is useful precisely because it gives confident, specific answers. A vague itinerary ('maybe visit some museums, food varies by area') is worthless. The confidence is the product.

But confidence without accuracy is worse than silence. And LLMs can't always distinguish between what they know well and what they're confabulating.

My current position is that the solution is layering. The AI handles experiential planning, the parts it's good at, that don't change, where being wrong has low stakes. Live data handles the operational facts, the parts where being wrong matters. The UI makes the distinction clear.

It's more engineering work. It would have been faster to just ship the pure AI output and add a disclaimer. But I've built apps for clients long enough to know that 'our AI might be wrong' is not a product strategy, it's an apology for not building properly.