Consumption, Delivery and Metering

The Grid’s infrastructure ensures that buyers of Instruments receive inference delivered in accordance with the benchmarks, and that tokens are properly metered and accounted for in consumption. This section explains how the underlying mechanism works.

The importance of accurate tracking

For each Instrument, buyers and suppliers rely on The Grid to:

• Track exactly how many tokens were delivered against each Unit.

• Confirm that performance stayed within the agreed instrument specifications.

• Maintain records to show rich usage metrics and resolve any disputes.

Canonical Usage Records

For every inference call, The Grid records usage data that includes:

• Timestamp

  The time of call and response from the API.

• Unit and Lot Information

  The Units from where tokens were deducted, and the Lots where the Units belonged.

• Token Counts

  Number of tokens input and output per call.

• Trade Details

  The individual trade(s) where the buyer purchased those Unit(s).

• Supplier Information

  The supplier that served the request.

• Performance Metrics

  Metrics such as Time to First Token and other measurements that are part of the Minimum Service Benchmark.

These records help us ensure that buyers receive the best quality inference, and that suppliers adhere with the Instrument Specifications at all times.

User Action Histories

While The Grid stores routing data records itself, users can log in into their accounts to find detailed histories of their actions:

• Order submissions and cancellations.

• Trades and associated price and size details.

• Transfers between Trading and Consumption.

• Sweep events and resulting Lots.

• Configuration changes and admin actions.