Inferock Bench is an open-source local proxy for tracking LLM cost. After pointing an SDK key and base URL at the proxy, it records tokens, truncated streams, empty responses, retries, and cache discounts per call and produces an independently reviewable receipt. Provider keys remain local and are not sent to Inferock servers.
Features
- Per-call cost receipts
- Failure and retry detection
- Multi-provider LLM compatibility
- Local key handling
Use Cases
- Audit LLM bills
- Find hidden retry costs
- Investigate anomalous API calls
- Compare realized model spend
FAQ
Inferock Bench is an open-source local proxy for tracking LLM cost. After pointing an SDK key and base URL at the proxy, it records tokens, truncated streams, empty responses, retries, and cache discounts per call and produces an independently reviewable receipt. Provider keys remain local and are not sent to Inferock servers. Core capabilities include: Per-call cost receipts, Failure and retry detection, Multi-provider LLM compatibility.
Common scenarios include: Audit LLM bills, Find hidden retry costs, Investigate anomalous API calls.
