Meter AI tokens

An AI proxy bills on tokens, not request counts. The pattern is a two-part loop: declare input and output token measures on one model_usage meter with the dimensions you price by (provider, model, modality, cache status, mode), price them in a catalog, bind the meter to the chat route with fs.meterRoutes(), then send the real per-request token counts from your backend with one report() call. The platform rates the signed measurement under the release it was admitted with.

This is the same meterRoutes + report() mechanism as Add metered routes, specialized for LLM usage. We build a small llm-api product to keep the example self-contained; it is the platform's enterprise-LLM golden sample.

Outcome#

The gateway settles the model's signed actual token counts on each successful response.

Prerequisites#

• A backend route that returns model usage
• A preview environment and runtime token
• A pricing decision in dollars per million tokens, per model and cache status

Declare the token meters and the chat route#

import * as fs from "@farthershore/business";

const requests = fs.requests();
const input = fs.measure("input_tokens");
const out = fs.measure("output_tokens");
const providerName = fs.dimension("provider");
const model = fs.dimension("model");
const modality = fs.dimension("modality");
const cacheStatus = fs.dimension("cache_status");
const mode = fs.dimension("mode");
const text = modality.value("text");
const uncached = cacheStatus.value("uncached");
const cached = cacheStatus.value("cached");
const acme = fs.provider("acme");
const acme4 = acme.model("acme-4");
const modelUsage = fs.meter("model_usage", {
  measures: [input, out],
  dimensions: [providerName, model, modality, cacheStatus, mode],
});
const llmPricing = fs.pricing("llm", {
  meter: modelUsage,
  catalog: [
    fs.rate.perMillion(fs.money.usd(3)).for(input, acme4, text, uncached),
    fs.rate.perMillion(fs.money.usd(15)).for(out, acme4, text, uncached),
    fs.rate.perMillion(fs.money.usd(0.3)).for(input, acme4, text, cached),
    fs.rate.perMillion(fs.money.usd(15)).for(out, acme4, text, cached),
    fs.modifier.multiplier(3, 2).when(mode.is("fast")),
  ],
});
const api = fs.backend("api", {
  transport: { mode: "direct" },
  meters: [requests, modelUsage],
  default: true,
});
const chat = fs.route("/v1/chat", {
  post: { backend: api, costs: [requests.fixed(1)], reports: [modelUsage] },
});

fs.meterRoutes("chat-model-usage", chat, {
  reports: [modelUsage],
  maxOutputUnits: out.atMost(8192),
});

fs.plan("enterprise", {
  kind: fs.plan.kind.usage,
  usagePricing: llmPricing.current(),
  grants: [chat],
  limits: [requests.perMinute(600)],
});

export default fs.business();

Cached input is $0.30/M instead of $3/M; output stays $15/M for both cache states; fast mode multiplies every matching charge by exactly 3/2. Rates are exact rationals end to end. maxOutputUnits: out.atMost(8192) bounds the output measure — the gateway clamps the client's max_tokens to 8,192 before signing the request upstream, so billing truncation and product truncation are the same event.

Report the real token counts from your backend#

Install @farthershore/backend, set FS_RUNTIME_TOKEN (see CLI quickstart for the agent workflow), verify the platform signature, then report the model's usage figures with ctx.report(). Before the response is sent it signs usage into the platform response path — no extra network call.

import { fartherShore } from "@farthershore/backend";

const fs = fartherShore.initFromEnv(); // derives everything from FS_RUNTIME_TOKEN

export async function POST(request: Request) {
  const url = new URL(request.url);
  const body = new Uint8Array(await request.clone().arrayBuffer());

  // Fail-closed verification: identity comes only from the platform's signature,
  // never from the plaintext X-FS-* headers.
  const ctx = await fs.verifyRequest({
    method: request.method,
    path: url.pathname,
    query: url.search,
    headers: request.headers,
    body,
  });

  const completion = await callModel(await request.json());

  await ctx.report({
    meter: "model_usage",
    values: {
      input_tokens: completion.usage.prompt_tokens,
      output_tokens: completion.usage.completion_tokens,
    },
    dims: {
      provider: "acme",
      model: "acme-4",
      modality: "text",
      cache_status: completion.usage.cached ? "cached" : "uncached",
      mode: "standard",
    },
  });

  return Response.json(completion);
}

The meter, measure, and dimension keys are plain strings that must match the release's measurement schema (model_usage, input_tokens, output_tokens, and the declared dimensions); an unknown key is rejected loudly at the gateway, never silently dropped. Values are validated locally before signing — they must be non-negative finite numbers. Identity is never an argument: it rides the verified context. Backends report measurements, never money.

Express upstreams#

If your upstream is Express, verify with the middleware and report the same way:

const fs = fartherShore.initFromEnv();
app.use(express.raw({ type: shouldCaptureRawBody, limit: "10mb" }));
app.use((req, _res, next) => {
  if (Buffer.isBuffer(req.body) && req.body.length > 0) {
    (req as typeof req & { rawBody?: Buffer }).rawBody = req.body;
  }
  next();
});
app.use(fs.middleware()); // strict and fail-closed by default
app.use(parseVerifiedJson); // parse only after signature verification

Use the generated backend scaffold's shouldCaptureRawBody and parseVerifiedJson helpers so streaming exemptions and size limits remain in sync with the SDK. Only fs.middleware({ always: false }) defers verification to a backend contract that explicitly disables it.

Build and verify#

farthershore build --format json

Push business/**, then drive a real chat request through a test persona and read the breakdown. Wait until env list contains the preview environment before creating its backend row. If automatic branch-prefix creation did not occur, create the preview explicitly first:

farthershore env list llm-api --format json
farthershore backend create llm-api --env <environment> \
  --name "Preview API" --slug api --transport direct \
  --idempotency-key <persisted-backend-create-attempt-key> \
  --origin-url https://preview-api.example.com --default --format json
farthershore backend list llm-api --format json
# Token meters appear per-dimension once traffic flows.
farthershore usage summary llm-api --format json

Filter the structured backend list by the preview's environment id.

Verify it works#

• farthershore build succeeds and the validated business lists model_usage with input_tokens + output_tokens measures.
• A POST /v1/chat call with max_tokens: 100000 is forwarded with max_tokens: 8192.
• input_tokens / output_tokens in the usage summary match the model's usage.prompt_tokens / usage.completion_tokens.
• The subscriber's bill preview rates a 1,000-uncached-input / 500-output call at exactly $0.003 + $0.0075 = $0.0105 (× 3/2 in fast mode).

Common failures#

• Usage stays at zero: the reported meter, measure, and dimension keys must exactly match the declared keys.
• Token totals double: call report() once per request; the SDK picks the channel.
• 422 admission_bound_exceeded: the client's max_tokens exceeds the declared bound and could not be rewritten.

Recover#

Stop test traffic, correct the meter bindings or backend report in preview, and re-run one request. Do not replay or adjust billable usage until the mismatch is understood.

Agent prompt#

Add a model_usage meter with input and output token measures and provider/model/modality/cache_status/mode dimensions to the existing chat route, price it per million tokens per model and cache status in a catalog, bind it with meterRoutes and maxOutputUnits, and report the model's exact counts through one report() call. Build and verify one preview request. Do not publish production or print runtime tokens.

Related#

• Add metered routes — the general meterRoutes + report() loop.
• Pricing catalogs — items, selectors, modifiers, tiers, quotes.
• Prepaid wallet — meter tokens down against a prepaid balance.
• CLI reference — mint FS_RUNTIME_TOKEN and a test persona via the CLI.