Batch Fill Rate Calculator — Throughput tokens/s derived from Requests, tokens. Sizing inference batches..
Requests is required
tokens is required
Batch Fill Rate Calculator — Throughput tokens/s derived from Requests, tokens. Sizing inference batches..
Formula
throughputTokensS = requests ÷ tokens