Worked solution · 11 min read
Design ad click infrastructure
Carry an impression id through to the click so attribution is a lookup rather than a guess. Log both sides as append-only streams, join in two tiers — a hot store for the recent window plus a batch reconciliation for late arrivals — and publish two numbers: a fast approximate one for dashboards and a reconciled one for billing.
One defensible approach, not a model answer or a marking scheme. No interviewer said any of this — the value is in the reasoning, so read it after attempting the question rather than instead of.
01Walkthrough
Nail the attribution window and the consumer
The window sets the whole storage strategy, and who consumes the output sets your accuracy requirement.
- A window of minutes can be held in memory. A window of days cannot, so the join must reach into storage.
- Billing consumers need reconciled exactness. Dashboards need speed and tolerate revision. Trying to serve both with one number is where these designs fail.
- Decide up front whether a click with no matching impression is dropped, counted, or quarantined.
Mint an impression id at serve time
When an ad is served, generate a signed id encoding the impression and carry it in the click URL. Attribution then becomes a key lookup rather than a heuristic match on user and timestamp — which is both cheaper and defensible when an advertiser questions a number.
- Sign or encrypt the id so a client cannot fabricate clicks for impressions that never happened.
- Embed the served timestamp so window expiry can be validated without a lookup.
- Include a client-generated idempotency key so a retried click collapses to one.
Log both sides append-only
Impressions and clicks go to separate partitioned logs, keyed by impression id. Neither is ever mutated. This is what makes the pipeline replayable, which matters because you will eventually need to recompute a day.
- Partition by time so window expiry is a partition drop rather than a scan.
- Impressions dwarf clicks by orders of magnitude, so size the two paths independently.
- Write locally then ship, so an ingest outage delays rather than loses data.
Join in two tiers
A hot key-value store holds the recent window for immediate attribution. A scheduled batch job then rejoins against the full impression log to catch anything the hot path missed — late clicks, clicks that arrived during an outage, ids evicted early.
- The hot tier gives near-real-time numbers and is allowed to be incomplete.
- The batch tier is the source of truth and produces the number you bill from.
- Emit the difference between the two as a metric. A widening gap is your earliest signal of a broken ingest path.
Publish two numbers, and say which is which
Fast-and-approximate for dashboards, reconciled-and-slower for billing. The mistake is not having two numbers; it is having two and pretending they are one.
- Label dashboard figures as provisional with a clear reconciliation time.
- Never revise a billed number silently. Restate explicitly, the same as any financial correction.
- Keep the mapping from every billed click back to an impression, so any charge can be explained.
Put fraud detection off the serving path
Fraud analysis needs cross-request context — patterns across a session, an IP range, a device — which is exactly what you cannot afford to compute while serving. Score asynchronously and exclude retroactively before billing closes.
- Cheap synchronous checks only: signature validity, window expiry, obvious replay.
- Expensive pattern detection runs on the stream, marking clicks as suspect rather than deleting them.
- Exclude suspect clicks at billing time, keeping them in the raw log so a decision can be revisited.
Trade-offs worth naming out loud
Saying what you rejected, and why, is most of what separates a senior answer from a correct one.
Explicit impression id carried in the click
instead of Probabilistic matching on user id and timestamp
Probabilistic matching needs no coordination with the serving path but produces numbers you cannot defend and quietly mis-attributes under load. An explicit id makes attribution exact.
Two-tier join
instead of Single streaming join over the full window
One streaming join is conceptually cleaner but needs the whole window in state, which is infeasible for a multi-day window at this volume. Two tiers cost complexity and buy tractable state.
Separate provisional and billed numbers
instead of One number for everything
One number means either dashboards wait for reconciliation or billing uses unreconciled data. Both are worse than being explicit about the difference.
What gets read as strong or weak
Strong signals
- Proposing a signed impression id unprompted, which handles both attribution and fabricated clicks at once.
- Recognising the window size decides whether the join can be in-memory at all.
- Separating provisional from reconciled numbers rather than promising exactness everywhere.
- Keeping fraud scoring off the serving path while still excluding it before billing.
Loses points
- Joining on user plus timestamp and calling it attribution.
- Claiming exactly-once end to end without saying where deduplication happens.
- Holding a multi-day window in streaming state without acknowledging the size.
- Deleting suspected fraud instead of marking it, leaving no way to revisit the call.