Bulk operations for a 1M-combination problem

The bottleneck

A human in the middle of every placement

Partners on KAYAK's business travel marketplace buy placement against routes. To change what they were buying, a partner submitted a request, and then waited, because every request landed in a manual review queue before anything went live.

The queue was doing real work: catching malformed bids, checking eligibility, preventing partners from buying things that didn't make sense. But it was doing that work serially, by hand, on roughly 1,500 requests a year, and the cost was paid entirely in partner waiting time. Every day a request sat in review was a day a partner wasn't spending.

The obvious fix was to make the review faster. I argued for removing it. A queue that exists to catch invalid configurations is a queue that exists because the system permits invalid configurations in the first place.

Removing the review

Moving validation into the flow itself

The redesign moved the validation from a person at the end to the structure of the flow itself. Partners select targeted routes inside a model that can't express an invalid state, so there's nothing left for a reviewer to catch. This is the bidless targeted-routes flow, where the constraint lives in the interface rather than in a downstream human.

The result was 1.29 days returned per request. It exceeded the KPI set for the project, and the number that actually matters is what that compounds to.

Bulk at scale

Targeted Routes V2.0

Removing the wait exposed the next problem: the flow was still one-at-a-time. A partner operating across a real route network was configuring placements individually, which meant the tooling scaled linearly with a business that didn't.

V2.0 restructured it around bulk operations: 500 placements per workflow, 1M+ combinations per request. The interaction problem there isn't the volume, it's comprehension at volume: at a million combinations nobody can review a list, so the design has to let someone specify intent, see the shape of what that produces, and catch the case where the shape is wrong before committing.

That's a fundamentally different design problem from anything else in this portfolio. There's no conversation, no model output, no ambiguity to resolve. There's a dense operational surface where the user is an expert, the stakes are financial, and the failure mode is a confident bulk action against the wrong set.

Model output as a decision

Optimizations

Partnering with ML engineering, I designed Optimizations, a bid-guidance surface that turns model output into something a partner can act on. The model produces a recommendation; the surface lets a partner model and compare the incremental impact of an operational change before committing spend.

The design principle is the same one that runs through the AI chat work, arrived at from the opposite direction: the system shouldn't assert an outcome it can't guarantee. A recommendation presented as a number reads as a promise. The same recommendation presented as a comparison between scenarios reads as what it is: guidance the partner is still deciding against.

The thing nobody asked for

An internal A/B testing platform

Reading experiment results at KAYAK meant going through an analyst. That was a bottleneck of exactly the same shape as the approval queue: a person in the middle of a loop that should have been self-serve, and it was slowing down every team, not just mine.

Nobody assigned this. I designed KAYAK's internal A/B testing platform so PMs and engineers could read experiment results directly, removing the analyst dependency from the iteration loop. It's the piece of work I point to when someone asks what I do that isn't in the brief.

What it returned

Time back to the people spending money