TPM and RPM rate limits

TPM means tokens per minute; RPM means requests per minute. Providers may enforce both, so either can throttle a workload.

Tiny example

Ten large requests might exhaust TPM before RPM. Thousands of tiny classification calls might hit RPM first.

FDE note

Use bounded concurrency, exponential backoff with jitter, provider-reported retry timing, and a queue or admission policy. A retry storm can make throttling worse. Track the applicable model, account, and regional limits because limits change.