KAYAK · 2025–2026 · Embedded conversational search
Moving KAYAK's AI into the booking funnel
KAYAK built a separate AI travel site. People tried it once and didn't come back. I rebuilt it as chat inside the flow where the booking happens, and proved the idea with one input field before anyone paid for the architecture.
- Role
- Product designer. I owned how the system reads a prompt, and the structure every answer follows.
- Team
- 1 PM, 1 staff designer, front-end and back-end engineering
- Timeline
- April 2025–April 2026
- Surfaces
- kayak.com, desktop and mobile web
The NLP Search experiment ran as an A/B test against the existing search. Booking revenue excludes ads. Every core funnel metric stayed neutral, so nothing we shipped cost anything.
At a glance
The problem
The AI ghost town
KAYAK put a year into a standalone AI travel site. Almost nobody found it, and the few who did never came back.
KAYAK.ai launched in April 2025 as a chat-first travel site, separate from the booking platform. It answered complicated travel questions and matched deals to narrow preferences, then handed the traveler to KAYAK.com to book.
Marketing spend and infrastructure spend both went in. Organic traffic stayed thin, and within a month of launch almost nobody was coming back.
Signals of opportunity
Then the referral data turned something up. The small group who did cross over to KAYAK.com clicked through at five times the rate of any other referral channel.
What the 5x doesn’t prove. Anyone who went looking for a separate AI travel site in 2025 was already an unusually motivated traveler, so the gap is at least partly selection and possibly all of it. A signal from a group that selected itself can justify an experiment. It can’t justify a platform rewrite.
KAYAK.ai’s technology wasn’t the problem. Asking users to go somewhere new for it was.
The bet
Testing the waters: NLP Search
The team wanted the full chat experience. I argued for one input field first, because a cheap test that fails costs a week.
If the barrier was the destination, a better KAYAK.ai was never going to fix it. The destination had to go.
How do we make KAYAK.ai better?
How do we get AI into the KAYAK people already use?
That gave us two things a test could actually answer.
Two testable hypotheses
- 01 Adoption AI features embedded in KAYAK.com get picked up at least as much as the standalone KAYAK.ai was.
- 02 Value People who engage with those features move core revenue, and don't cost the existing funnel anything.
AI chat vs. AI search mode
The long-term goal, and the thing we could ship in weeks
- AI mode Tested first One more mode next to Flights, Stays and Cars. A single freeform input, routed to the results page KAYAK already had.
- AI chat drawer A persistent panel beside the results. Multi-turn, new components, new interaction patterns. The version everyone wanted.
A persistent chat meant new components, a migration and real dev cost, and if it failed the learning would be hard to separate from everything else that changed with it.
Letting people type plain language into the existing search form looked cheaper until we costed it. That form routes to structured queries, so it would need a new backend path deciding, on every keystroke, whether someone is writing a sentence or an airport code. A separate AI mode avoided that and isolated the risk. If the experiment broke, only the experiment broke.
The test
Plain language, typed into the core search flow, landing on the results page KAYAK already had.
One input field, inside the search flow people already used. The cheapest way to find out whether the thesis held.
Behavior before pixels
Prompt engineering: classify or infer
The hard problem was never what the AI looked like. It was how the system decides what a traveler is asking for.
A prompt goes in and an API call comes back. Everything between the model's answer and a response a traveler can act on was the open question.
We looked at two ways to turn a prompt into a response.
Parametric classifier
- Book now Scenario 1
- Explore deals Scenario 2
- Inspirations Scenario 3
- Find best time Scenario 4
- General explore Scenario 5
- Daydreaming? Scenario 6
- Price hesitant Scenario 7
- Content browse Scenario 8
- Not traveling Scenario 9
Prompt examples
- "Plan a Europe trip" → Scenario 8
- "Visa tips for Japan" → Scenario 7?
Add hotels, cars and activities and the grid multiplies, 3 × 3 × 3 × … exponential complexity.
Nine fixed scenarios, read off how complete the dates and the destination are. Fully specifiable, easy for engineering to estimate, and it would have broken on the first prompt that didn’t think in search parameters.
Intent inference engine
- Travel searchHigh intent users Result pageBook flights, hotels, cars
- Destination searchMid intent users Explore pageBrowse, get inspired
- General questionLow intent users LLM answerTravel advice, info
- Plan across verticalsPower users LLM answer + trip planIn-depth tool
Prompt examples
- "Flights to Boston early next month"
- "Summer family trip to Europe"
- "Is my flight delayed?"
- "Plan a 5-day trip to Italy"
So the model infers what a traveler is trying to do, sorts it into a rough bucket, then resolves the ambiguity inside that bucket with one follow-up question. A vague prompt asks for one more thing and never dead-ends.
Simulate user flow in a custom GPT
I mapped the logic in Figma as a jobs-to-be-done flow, down to the fallbacks, then translated it into a custom GPT so the behavior could be run.
A Figma file gets reviewed. A GPT gets used. Engineering and product typed real prompts and watched the system answer, which iterated the system prompt faster than a review cycle and cost no dev time.
That simulation became the behavioral contract between engineering and product before a single mock existed.
The evidence
Green light to the final lap
+14% booking revenue from one input field, and a ceiling that told us exactly what to build next.
What the experiment returned
- +14% Real booking revenue, excluding ads
- +2% Lift in total conversion
- Neutral Impact on core metrics. No downside risk materialized.
On the gap between the first two numbers. Conversion moved 2% and revenue moved 14%, so the lift came more from what people booked than from how many of them booked. We didn’t decompose it at the time. My best explanation is that AI-assisted sessions skewed toward longer and multi-vertical itineraries.
The ceiling was in the same data. People submitted one query and stopped. There was nothing to refine and no way to follow up, because the architecture was single-turn by design.
Building it out
Scaling to a chat drawer
Chat went site-wide in a drawer KAYAK already had, so there was no new interaction model to teach.
Left vs. right?
Where the panel sits tells people what the AI is for. Left panels, the pattern in Lovable and ChatGPT Canvas, read as a creative partner, prompted in rounds. Right panels read as a contextual assistant that stays out of the way.
Panel placement is a claim about the AI's role
- Left panel Reads as a creative partner. Rounds of prompting, co-creating the artifact the model produces.
- Right panel Chosen Reads as a contextual assistant. Results keep the attention, chat assists on demand.
Integrating with existing architecture
KAYAK’s Trips drawer was already on the right. Reusing it meant no new interaction model, and it put chat next to the place people save a trip.
What went wrong
Three problems in one reply
One flight reply showed three separate failures at once, and only one of them was easy.
- Unstable layout. The summary is the last thing the model generates and it rendered at the top, so every answer reflowed the moment results arrived.
- Lost in the panel. The full results list was crammed into the narrow chat column. Fullstory replays showed people scrolling the results away and losing the page behind them.
- Orphaned saving. The save control sat inside chat, cut off from where people actually manage a trip.
Saving was the easy one. It came out of chat and the Trips drawer took it back. The other two are the next two sections.
Craft decisions
Defining a coherent response
When the answer can be anything, the layout can't be one thing. I fixed the order instead of the shape.
Multi-turn chat made the output non-deterministic. One prompt might return a list, a comparison, a clarifying question, or a plan across three verticals, and with the summary landing last and rendering first, every answer reflowed.
So I stopped designing a layout and defined a fixed response anatomy, an order any answer pours into. Status and acknowledgement hold the wait, then the persistent block settles, widget first, summary into a slot reserved before it exists.
That order also controls cost. Status and acknowledgement are static strings with no per-turn model call, so holding the wait stays free no matter how long a conversation runs.
One slot stayed loose on purpose. Result widgets are variable and optional, and what each one should actually be was the next problem.
The research
Chat as a navigation layer
Nine people told us what the panel was for. The majority said it was a route, not a destination.
The backend had unlocked multi-vertical search, so one prompt could now return several result pages. Product wanted engagement up. And the panel still had to route people out to results without adding friction to the core search.
What users actually want from chat
I tested three widget patterns with nine people, unmoderated, through usertesting.com. Pattern B, a condensed summary with thumbnails, won.
Two of the three patterns tested
- Pattern A · full list The results rendered inside the panel. Everything you need is here, and it runs the length of the column.
- Pattern B · condensed Chosen Route, dates, airline marks, a price to anchor on, a viewing badge. Enough to pick a page and act on it.
The scorecard was the small win. The real finding was how differently people framed the panel.
- Reference Builder 0 of 9 Front-loads the trip and the constraints into chat, then hands that understanding to conventional search. Hypothesis · untested
- Chat-as-Destination 2 of 9 Expects to compare and act inside the chat, without going out to a results page.
- Results-Page Purist 2 of 9 Came for the results page and wants it left alone. The chat is an afterthought.
- Chat-as-Wayfinder 5 of 9 Chat is the fastest way to the right page. Somewhere to pass through on the way to acting. The direction we took
What nine people can and can’t decide. Nine unmoderated participants is discovery-grade evidence and this was a platform-grade decision. It agreed with the single-turn ceiling in the experiment data and with those same Fullstory replays. Three signals pointing the same way is what made the call.
The Reference Builder is still a hypothesis. It kept suggesting itself in the sessions and no participant cleanly matched it, so it went on the list to test.
Building a search vertical with an interactive component library
Nothing renders results inside the chat. Each response generates a vertical widget: a compact anchor in the panel, linked to a results page on the core content area where people already know how to act.
One widget, one page, one state. Apply a filter and the new URL generates a new widget, and each one carries a viewing badge so a traveler can tell which pages they have opened.
The strongest delight signal in the research was that mixed prompt. “I don’t think all chatbots would offer that.”
Two things went on the list for after the /ai migration: widgets that adapt to the chat mode, for the people who want to act inside the panel, and a collapsible resizable panel for the ones treating it as a reference.
What it added up to
Reflections
+123% engagement, and a number underneath it that I would defend harder than the headline.
Chat v1, measured against NLP Search
- +123% User engagement
- +80% Messages per user
The honest caveat on the first number. Comparing a multi-turn chat product to a single-turn search input is close to comparing a conversation to a sentence. The artifact was built to produce more turns, so more turns is a weak claim. Messages per user is the one I’d defend. It says the people who started a conversation kept going, which is the specific thing the single-turn ceiling said we couldn’t do.
Key takeaways
What I'd carry into the next one
- 01 Ask before answering One clarifying step costs a couple of seconds and saves a traveler from a dead-end session.
- 02 Classify the intent, not the query A deal hunter and a trip planner can type the same words and need completely different answers.
- 03 Replace recaps with actions Seven of nine people called the text summaries noise. Proactive follow-ups beat them every time.
- 04 Prototype with real data Static mocks can't test an intent-driven system. You have to design for outputs you can't predict.
The most valuable work sat upstream of the UI, in the behavioral spec and in the research behind it. Once those clicked, the pixels followed.