Per-tenant token budgets in NestJS
Nest's interceptors and guards are exactly the right shape for cost control: measure in one place, enforce before the handler runs, and no feature can be added without both.
The economics are language-independent — what tokens cost and where the money goes is the model, and the Node page covers the runtime mechanics.
What Nest adds is placement. Cost control has two halves — measure everything and refuse before spending — and Nest already has a first-class home for each: an interceptor and a guard. Put them there and a new feature cannot ship unmeasured, because the plumbing is global.
Attribution: AsyncLocalStorage, not a constructor parameter#
Every LLM call needs to be tagged with tenant, user and route. Threading that through service constructors is how you end up with a request-scoped provider, which is contagious and slow.
import { AsyncLocalStorage } from 'node:async_hooks';
export interface CostContext {
tenantId: string;
userId?: string;
route: string;
requestId: string;
}
export const costStore = new AsyncLocalStorage<CostContext>();@Injectable()
export class CostContextMiddleware implements NestMiddleware {
use(req: Request, _res: Response, next: NextFunction) {
costStore.run(
{
tenantId: (req as any).user?.tenantId ?? 'anonymous',
userId: (req as any).user?.id,
route: req.route?.path ?? req.path,
requestId: (req.headers['x-request-id'] as string) ?? randomUUID(),
},
() => next(),
);
}
}export class AppModule implements NestModule {
configure(c: MiddlewareConsumer) {
c.apply(CostContextMiddleware).forRoutes('*');
}
}Singleton services throughout, full attribution, no scope contagion.
The quota guard: refuse before you spend#
A guard runs before the handler, which is precisely where a budget check belongs. An over-quota tenant should never reach the code that makes the call.
export const QUOTA = 'quota';
export const Quota = (feature: Feature) => SetMetadata(QUOTA, feature);
@Injectable()
export class QuotaGuard implements CanActivate {
constructor(
private readonly reflector: Reflector,
private readonly ledger: LedgerService,
) {}
async canActivate(ctx: ExecutionContext): Promise<boolean> {
const feature = this.reflector.getAllAndOverride<Feature>(QUOTA, [
ctx.getHandler(), ctx.getClass(),
]);
if (!feature) return true; // not an LLM route
const { tenantId } = costStore.getStore() ?? { tenantId: 'anonymous' };
const { spentMicros, capMicros } = await this.ledger.usage(tenantId);
if (spentMicros >= capMicros) {
throw new HttpException(
{
statusCode: 429,
error: 'Quota exceeded',
message: 'Monthly AI usage limit reached.',
resetsAt: this.ledger.periodEnd(tenantId),
},
HttpStatus.TOO_MANY_REQUESTS,
);
}
return true;
}
}@Quota('reply_draft')
@Post('drafts')
create(@Body() dto: CreateDraftDto) {
return this.drafts.create(dto);
}429 with a resetsAt is the right response — it tells the client this is a quota problem and when it clears, rather than a generic failure they will retry into.
The interceptor: measure everything#
An interceptor wraps the whole handler, so it sees the outcome, the timing and — with a small collector — the usage from every model call made inside it.
@Injectable()
export class CostInterceptor implements NestInterceptor {
constructor(private readonly ledger: LedgerService) {}
intercept(ctx: ExecutionContext, next: CallHandler): Observable<unknown> {
const started = Date.now();
const collector = new UsageCollector();
return usageStore.run(collector, () =>
next.handle().pipe(
tap({
next: () => this.flush(collector, started, 'ok'),
error: () => this.flush(collector, started, 'error'),
}),
finalize(() => {
// fires on client disconnect too — the abandoned-stream case
if (!collector.flushed) this.flush(collector, started, 'aborted');
}),
),
);
}
private flush(c: UsageCollector, started: number, outcome: Outcome) {
c.flushed = true;
const ctx = costStore.getStore();
void this.ledger.record({
...ctx,
calls: c.calls,
inputTokens: c.input,
cachedTokens: c.cached,
outputTokens: c.output,
costMicros: c.costMicros,
latencyMs: Date.now() - started,
outcome,
});
}
}finalize is the important operator. It runs on success, on error and on unsubscribe — which is what happens when the client disconnects mid-stream. Without it, abandoned requests are the one category of spend that never reaches your ledger, and they are exactly the category you most want to see.
Register it globally so no route can be added without accounting:
providers: [{ provide: APP_INTERCEPTOR, useClass: CostInterceptor }]Cancel upstream when the client leaves#
The interceptor records the abandonment. This stops it costing money in the first place.
@Quota('chat')
@Sse('stream')
stream(@Body() dto: ChatDto, @Req() req: Request): Observable<MessageEvent> {
const ac = new AbortController();
req.on('close', () => ac.abort());
return new Observable<MessageEvent>((subscriber) => {
void (async () => {
try {
for await (const chunk of this.llm.stream(dto, { signal: ac.signal })) {
subscriber.next({ data: chunk.text });
}
subscriber.complete();
} catch (err) {
if ((err as Error).name !== 'AbortError') subscriber.error(err);
else subscriber.complete();
}
})();
return () => ac.abort(); // unsubscribe also cancels upstream
});
}Both paths abort: the raw close event and the Observable teardown. Belt and braces here is justified, because the failure is silent and shows up only on the bill.
The ledger#
Two writes, deliberately: a durable row for reporting, and a fast counter for the guard to read.
@Injectable()
export class LedgerService {
constructor(
private readonly prisma: PrismaService,
@Inject(CACHE_MANAGER) private readonly cache: Cache,
) {}
async record(entry: LedgerEntry): Promise<void> {
await this.prisma.llmSpend.create({ data: entry }); // durable, for reports
const key = `spend:${entry.tenantId}:${period()}`;
await this.cache.set(key, ((await this.cache.get<number>(key)) ?? 0) + entry.costMicros);
}
async usage(tenantId: string) {
const key = `spend:${tenantId}:${period()}`;
const cached = await this.cache.get<number>(key);
if (cached !== undefined) return { spentMicros: cached, capMicros: await this.cap(tenantId) };
const agg = await this.prisma.llmSpend.aggregate({
where: { tenantId, createdAt: { gte: periodStart() } },
_sum: { costMicros: true },
});
const spent = agg._sum.costMicros ?? 0;
await this.cache.set(key, spent);
return { spentMicros: spent, capMicros: await this.cap(tenantId) };
}
}costMicros is an integer. Never a float — this is money in a billing path, and floats lose fractions across millions of rows.
The guard reads the cached counter, so a quota check costs a Redis GET rather than an aggregate query on every request. The cache is a performance optimisation over a durable source of truth, which means a cache flush degrades into a slow request, not a wrong one.
The report#
@Roles(Role.Admin)
@Get('admin/spend')
async spend(@Query() q: SpendQueryDto) {
return this.prisma.llmSpend.groupBy({
by: ['feature', 'model'],
where: { createdAt: { gte: q.since } },
_sum: { costMicros: true, inputTokens: true, cachedTokens: true, outputTokens: true },
_count: true,
orderBy: { _sum: { costMicros: 'desc' } },
});
}Three numbers to derive from it and watch:
- Cache hit rate =
cachedTokens / (inputTokens). High and flat. A drop means someone put something volatile at the top of a prompt. - Cost per successful call, not per call — the failed ones still cost input tokens.
- Abort rate. Climbing means answers are too slow or too long, and both are money.
Why Nest is a good fit for this
Because the two halves of cost control map onto framework primitives that are already global. A guard that refuses before the handler runs, and an interceptor that measures whatever happens — including the disconnect. Register both with APP_GUARD and APP_INTERCEPTOR and a new endpoint is metered and capped by default, with the developer having to opt out rather than remember to opt in.
Common questions#
Guard or interceptor for the budget check?#
Guard. It runs before the handler, so an over-quota request never reaches the code that spends money, and it can return a proper 429 without the handler knowing quotas exist. The interceptor's job is measurement, which has to wrap the handler to see the outcome.
Why not a request-scoped provider to hold the cost context?#
Because Scope.REQUEST is contagious: everything that injects it becomes request-scoped too, all the way up to the controller, and you pay for constructing that chain on every request. AsyncLocalStorage gives you the same per-request value with singleton providers.
Should the quota reset monthly or roll?#
A rolling window is fairer and harder to game; a calendar month is easier to explain on an invoice and to implement. Start with the calendar month, key your cache by period, and revisit if customers complain about the cliff at month end.
How do I handle a tenant who legitimately needs more?#
Make the cap a column, not a constant, and let support raise it — that is the whole reason cap() is a lookup rather than a config value. A hard-coded limit means every exception is a deploy.
Get the NestJS agent pack
A battle-tested AGENTS.md, the review checklist, and the failure-mode cheat sheet for NestJS. One email, then occasional updates when the tooling shifts. No course pitch.
AGENTS.md now — no email needed.