Design GPU billing for a distributed cluster

Whether you treat billing as a correctness problem rather than a reporting one. Analytics can lose a row and nobody notices; a billing pipeline that loses or duplicates one produces a wrong invoice, and someone disputes it.

• What granularity does billing need — per second, per minute, or per hour?
• Who absorbs the cost when a node dies mid-job: the tenant or the platform?
• Are GPUs billed on allocation or on measured utilisation?
• How long after month end can an invoice still be corrected?

Roughly 5,000 GPUs across three regions, 200 tenants, jobs from seconds to multi-day, invoices cut monthly. That is a few hundred million metering events a month — small enough to store, large enough that a 0.1% duplication rate is a visible billing error.

1. 01Emit usage events at the source, each carrying a job id, tenant, GPU id, and a monotonic interval — not a running total.
2. 02Give every event a deterministic id so the same interval reported twice collapses to one row.
3. 03Aggregate into per-tenant, per-period buckets that can be recomputed from raw events rather than mutated in place.
4. 04Treat invoicing as a snapshot over a closed period, keeping the raw events so any line item can be explained.
5. 05Decide the preemption rule explicitly and make it visible on the invoice.

An outline, not a model answer. Interviewers are listening for how you reason, so working through it yourself beats reciting this.

• A metering agent is partitioned for two hours and then flushes its backlog — what happens?
• A customer disputes a line item three weeks after the invoice. How do you reconstruct it?
• Usage from a region arrives after the period closes. Amend, or roll into next month?

• Aggregating counters in place, leaving no way to recompute or audit a total.
• Assuming events arrive exactly once, so a retry silently double-bills.
• Designing the happy path and treating preemption and late arrivals as edge cases, when they are the substance of the question.