What Is TPM (Tokens Per Minute)? The Truth Behind Rate Limiting Triggered by a Single Extra-Long Text Input
Why does it say I exceeded the limit when I only made one call in a minute? This deep dive explains the TPM (tokens per minute) mechanism and the hidden rate-limiting pain point caused by dragging in a million-character document in a single request.
1. The Big Pitfall Many People Fall Into by Looking Only at RPM
RPM counts the [number of requests], while TPM counts the [token throughput consumed per minute].
Even if you initiate only 1 request per minute (far below the 15 RPM request cap), if that single request directly drags in a huge PDF file containing 500,000 characters (i.e., 700,000 tokens), your single request instantly exceeds the per-minute token threshold allocated to you by the system, causing the system to directly refuse to respond.
2. The Solution
When processing massive documents, use streaming chunking or the official Context Caching (context caching) technology to smooth out a one-time, highly bursty token throughput.