Do you know how many calls your agent fleet will make against Salesforce in the ten minutes after a downstream timeout starts? Most teams find out from the vendor's error codes rather than from their own dashboards.
In a pilot, three agents and a handful of test users rarely come close to an API quota. That said, the same workflows promoted to production — forty agent instances, overnight batch enrichment, and a retry policy nobody revisited — can consume a day's API allocation before the morning shift logs in.
This post covers the mechanics that keep that from happening: queueing, token-bucket budgets per agent identity, circuit breakers, and graceful degradation. Each one protects the systems of record your business actually runs on, and each one belongs in the architecture before the first production cutover, not after the first incident.
Agent backpressure is the set of controls that slow, queue or shed agent traffic before it exceeds what a rate-limited system of record can absorb. It protects Salesforce, ServiceNow or Workday from the agents calling them.
Why Agent Fleets Hit Rate Limits Harder Than Human Users
A human user in Zendesk makes a few dozen API-backed requests an hour, spaced out by reading, typing and thinking. An agent has none of that natural pacing, and it will issue its next tool call the moment the model returns one.
What's more, agents compound load in ways that are easy to miss in a design review. The three patterns we see most often in production are worth naming specifically:
- Fan-out. A single triage request spawns a lookup agent, an enrichment agent and a summarization agent, each of which reads the same account, contact and case records independently. One logical task becomes nine or twelve API calls.
- Stacked retries. The HTTP client retries, the tool wrapper retries, and the orchestration framework retries the whole step. Each layer was configured reasonably in isolation, and together they multiply.
- Synchronized schedules. Batch jobs kicked off by EventBridge at the top of the hour all start at once. As a result, the vendor sees a burst rather than a steady rate.
All of these behave fine at pilot scale, which is exactly why they survive into production. For a broader view of what changes at that transition, see our breakdown of moving AI agents from pilot to production.
The retry multiplication math. Three layers, each configured for three retries, means up to four attempts per layer. That is 4 × 4 × 4 = 64 calls against the system of record for one logical operation that was never going to succeed while the vendor was throttling.
Keep in mind that most systems of record enforce more than one limit at a time. Salesforce, for instance, tracks a rolling 24-hour request allocation and returns REQUEST_LIMIT_EXCEEDED when it is exhausted, while also capping concurrent long-running requests; HubSpot enforces both a burst window and a daily ceiling. An agent that respects one limit can still trip the other.
What Does Backpressure Actually Mean For An Agent Fleet?
In stream processing, backpressure means a slow consumer signals upstream producers to slow down. For agent fleets, the system of record is the slow consumer, and the agents are producers that, left alone, will never slow down on their own.
For agents, backpressure means the platform — not the model — decides when a tool call runs. Calls wait in a queue, draw from a per-identity budget, and stop entirely when a circuit breaker opens.
The key architectural decision is where that control lives. It cannot live in the prompt, because a model instructed to "be careful with API usage" has no visibility into quota state across forty concurrent instances.
Accordingly, backpressure belongs in the tool execution layer — the gateway or tool registry that every agent call passes through. This is the same chokepoint where you enforce scoped credentials, which is why backpressure and agent identity and access controls are usually built together.
Queueing: Put A Buffer Between Reasoning And The System Of Record
The first control is the simplest one. Write operations against systems of record should go through a durable queue rather than straight from the agent to the vendor API.
On AWS, that typically means SQS in front of a Lambda consumer, with the event source mapping's maximum concurrency set to a number derived from the vendor's limit rather than from Lambda's defaults. A queue absorbs bursts, smooths synchronized schedules, and gives you a single place to measure how much work is waiting.
Queueing also changes how the agent experiences a slowdown. Instead of receiving a 429 and deciding on its own what to do, the agent receives an acknowledgment with a work ID and moves on, while the consumer drains at a rate the vendor can sustain.
- Idempotency keys on every message. SQS standard queues deliver at least once, so the consumer must recognize a duplicate CRM update. Derive the key from the task ID and step, not from a timestamp.
- A dead-letter queue with a real owner. Messages that fail after the retry policy is exhausted land in a DLQ that someone actually reviews. A DLQ nobody watches is a silent data-loss path.
- Priority separation. Customer-facing work and overnight enrichment go on separate queues with separate consumers. Otherwise, a large backfill can starve a live support workflow.
- Age-of-oldest-message alarms. Queue depth tells you volume, while message age tells you whether the business is waiting. Alarm on age.
Note that queueing is not appropriate for every call. Reads that the agent needs in order to reason — pulling an account record before drafting a reply — usually need to be synchronous, which is where budgets come in.
Token-Bucket Budgets Per Agent Identity
A token bucket is a counter with a capacity and a refill rate. Each API call removes a token; if the bucket is empty, the call waits or is rejected; tokens refill at a steady rate up to the capacity, which permits short bursts without allowing sustained overload.
Give each agent identity its own token bucket sized from the vendor quota. One runaway agent then drains its own budget and stalls, instead of exhausting the shared quota that humans and integrations need.
The phrase "per agent identity" matters more than the algorithm. If every agent shares one integration user and one global limiter, a single looping agent can consume the entire allocation, and your logs cannot tell you which one did it.
To illustrate how the sizing works, suppose a Salesforce org has a rolling 24-hour allocation of 500,000 calls. A reasonable starting split might look like this:
- Human users and existing integrations: 40%. That is 200,000 calls held in reserve that agents can never touch. Your MuleSoft jobs and the sales team's browser sessions were there first.
- Customer-facing agents: 35%. Roughly 175,000 calls, divided across agent identities by expected volume, with burst capacity set high enough for a busy hour.
- Batch and enrichment agents: 15%. About 75,000 calls, refilled slowly, because overnight work can tolerate waiting.
- Unallocated headroom: 10%. The remaining 50,000 calls absorb estimation error and incident recovery traffic.
Implementation detail matters here. The bucket state has to be shared across every instance of a given agent, so it lives in Redis — with the check-and-decrement done atomically in a Lua script — or in DynamoDB using conditional writes; an in-memory limiter per Lambda container does not enforce anything at fleet scale.
Furthermore, it pays to meter in the vendor's own units. Some APIs count a composite or bulk request as a single call, while others weight requests by cost, so a bucket that counts HTTP requests can drift badly from what the vendor is actually charging against your quota.
Circuit Breakers That Know The Difference Between 429 And 500
Budgets assume you know the limit in advance. Circuit breakers handle the case where the vendor pushes back anyway — because of a limit you did not model, a shared quota consumed by another team, or an outage on their side.
A circuit breaker has three states. Closed means calls flow normally; open means calls fail immediately without touching the vendor; half-open means a small number of probe calls are allowed through to test whether the system has recovered.
Trip the breaker on throttling signals — 429s, Retry-After headers, vendor limit codes — and on sustained 5xx errors. Then fail fast while it is open, so agents stop adding load to a system already struggling.
The common mistake is treating every error the same way. A 429 with a Retry-After header is the vendor telling you exactly when to come back, whereas a 400 for a malformed payload will fail no matter how long you wait and should never count toward tripping the breaker.
Here is how the main response classes should map to breaker behavior:
| Response | What it means | Breaker action | Retry? |
|---|---|---|---|
| 429 with Retry-After | Explicit throttling | Count toward trip; honor the header as minimum open time | Yes, after the stated delay |
| Vendor limit code (e.g., REQUEST_LIMIT_EXCEEDED) | Daily or rolling allocation exhausted | Open immediately; hold until the allocation window rolls | No, until reset |
| 503 / 502 / 504 | Vendor or network degradation | Count toward trip on a sliding window | Yes, with jittered backoff |
| 400 / 422 | Bad request from the agent | Do not count; route to the DLQ | No |
| 401 / 403 | Credential or scope problem | Do not count; page the owner | No |
Scope matters as well. A breaker per vendor endpoint — Salesforce composite API separate from Salesforce bulk API, ServiceNow Table API separate from its import sets — prevents one degraded surface from shutting down healthy ones.
Retries Without The Retry Storm
Retries are where most agent-driven overload actually originates. Therefore, the single most valuable change is usually to remove retries from every layer except one.
Pick the tool execution layer as the only place that retries, and set the HTTP client and the orchestration framework to zero. Then apply exponential backoff with full jitter, so forty agents that failed at the same moment do not all retry at the same moment.
In addition, cap retries with a retry budget rather than only a per-call count. A common pattern allows retries to make up no more than 10% of a given identity's traffic over a sliding window; once that ratio is exceeded, new failures go straight to the queue or the DLQ.
This also applies to model calls. Bedrock returns a ThrottlingException when you exceed on-demand quotas, and an agent loop that retries the model aggressively will also re-run every downstream tool call that follows, so model throttling and API throttling need to be governed together. We cover the guardrail and retry patterns behind this in reliability engineering for regulated AI.
Graceful Degradation: Decide What The Agent Does When It Cannot Write
When a breaker opens or a budget runs dry, the agent still has a user waiting. Graceful degradation is the set of pre-decided fallbacks that keep the workflow useful without adding load to the system of record.
Degradation modes are fallbacks chosen in advance: read from a cache, queue the write for later, switch to draft-for-approval, or shed low-priority work. The agent picks from that list and never improvises.
- Serve reads from a replica. Account and case data synced into Snowflake or Postgres on a short interval can answer most lookup questions. The agent labels the data with its sync time so the user knows it may be minutes old.
- Defer writes. The agent completes its reasoning, enqueues the update with an idempotency key, and tells the user the change is pending. This works well for CRM field updates and ticket notes.
- Drop to draft mode. The agent prepares the change and hands it to a human for approval instead of executing it. This is the right fallback for anything financial or customer-visible.
- Shed batch work. Enrichment and backfill agents pause entirely so their budget can be reassigned. Overnight jobs can always run again tomorrow.
Remember that a degradation mode is only safe if the stale-state read it implies is acceptable for that workflow. An agent answering "what tier is this customer on" from a fifteen-minute-old replica is fine, while one approving a refund against a balance that may have changed is not.
Of course, the handoff to a human in draft mode needs the same care as any other handoff. The context packaging is covered in our guide to agent handoff patterns.
Bounding Fan-Out By Construction
Budgets and breakers limit how fast calls go out. It is equally worth limiting how many calls a single task is allowed to generate in the first place.
Set a hard ceiling on sub-agent spawns and total tool calls per task, enforced by the orchestrator rather than requested in the prompt. A task that hits its ceiling stops and escalates, which converts a potential runaway into a bounded, observable failure — the same principle we develop in bounding multi-agent autonomy by construction.
Moreover, deduplicate reads within a task. If three sub-agents need the same account record, a per-task read cache keyed on object ID and version cuts those three calls to one at essentially no cost.
How Do You Know Your Backpressure Is Working?
Backpressure that is never measured tends to drift. Vendor limits change with contract renewals, new agents get added without budgets, and the headroom you reserved quietly disappears.
Track quota consumption by agent identity, breaker state changes, queue message age, retry ratio and degradation-mode entries. If you cannot attribute a vendor 429 to a specific agent, the system is not yet working.
- Quota consumption by identity, as a percentage of allocation. Pull the vendor's own usage figures where an API exposes them, and reconcile against your bucket counters daily.
- Breaker transitions. Every closed-to-open event should emit a structured log line with the endpoint, identity and triggering response code.
- P95 queue age per priority class. Rising age in the customer-facing queue is an early warning well before any user complains.
- Time spent in degraded modes. If an agent spends 20% of its week in draft mode, the budget is mis-sized, not the agent.
These signals feed naturally into the service-level objectives you set for the fleet. For the instrumentation itself, see our guidance on agent observability, and for turning these signals into targets, see SLOs for production agents.
Finally, rehearse the failure. Use a sandbox org or a fault-injection proxy that returns 429s on a schedule, and confirm that breakers open, queues absorb, and agents enter the right degradation mode — then fold the procedure into your agent incident response runbook.
Rolling It Out Without Breaking The Pilot
Teams that already have agents in production rarely get to rebuild from scratch. The good news is that these controls can be layered in incrementally, in a sequence that reduces risk at every step.
First, run the token buckets in shadow mode: count and log what would have been rejected, without rejecting anything. Next, consolidate retries into a single layer, since that change alone typically removes most of the amplification.
Then add circuit breakers per endpoint, followed by write queues for the highest-volume workflows. Once those are stable, enforce the budgets, starting with batch agents where a mis-sized bucket costs the least.
Overall, the goal is that the systems of record never have to be the ones to say no. When Salesforce, ServiceNow or Workday starts throttling, your agent platform should already have slowed down, queued the work, and told the user exactly what is pending.
Frequently Asked Questions
Should backpressure live in an API gateway or inside the agent framework?
Put it in a shared tool execution layer or gateway that every agent call passes through. Framework-level limiters only see one process, so they cannot enforce a fleet-wide quota across concurrent instances.
How do you size a token bucket when the vendor limit is undocumented?
Start conservatively, run the limiter in shadow mode for two weeks, and log every 429 with its headers. Many vendors return remaining-quota headers you can use to calibrate capacity and refill rate from real traffic.
Do agents need separate credentials for per-identity budgets to work?
Yes, in practice. A dedicated integration user or OAuth client per agent role lets you attribute vendor-side usage and revoke one agent without touching the others. Shared credentials make per-agent metering guesswork.
What is the difference between a retry budget and a retry count?
A retry count caps attempts per call. A retry budget caps retries as a share of an identity's total traffic, such as 10%, so a widespread failure stops generating retries fleet-wide instead of multiplying them.
Is queueing safe for writes that the user expects to see immediately?
Only if the agent says the change is pending and the queue drains within an agreed time. For writes a user must confirm in real time, keep them synchronous and protect them with a budget and breaker instead.
Get A Second Set Of Eyes On Your Fleet
If your agents are about to move from pilot traffic to production volume against Salesforce, ServiceNow or a homegrown ERP, the team at iSimplifyMe builds and operates production agent systems across CRM, ticketing and data warehouse environments every week.
Our AI agent operations work starts with a working session: we will map every system of record your agents touch, size budgets against your actual vendor quotas, and leave you with a backpressure design you can deploy in shadow mode the following sprint.