Who gets paged at 2:14 a.m. when your claims-intake agent starts writing duplicate records into Salesforce? If your team needs more than a few seconds to answer, the deployment decision in front of you is really an operations decision, and a managed agent engagement exists to settle it.
Most enterprises can get an agent to a working demo on Bedrock or the Claude API in a few weeks. That said, most of an agent's cost comes in the months after launch — model retirements, schema drift in the systems it touches, retry storms, audit requests — and a managed deployment contract has to cover that period in writing.
A managed AI agent deployment means an outside team builds, runs and is on call for your agents inside your own AWS account, under written SLAs, a defined handoff, and data boundaries your security team can audit.
This guide covers what that engagement should include, which contract terms are worth negotiating, and the criteria infrastructure leaders use to compare managed, in-house and hybrid models. For the wider discipline, our overview of AI agent operations sets the context.
What Does A Managed Agent Deployment Include?
A credible engagement covers three phases, and each one has its own deliverables. Keep in mind that a proposal covering only the first phase is a build contract with a support email attached.
Build
The build phase turns a workflow into a production system with the controls already in place. Deliverables for this phase include but are not limited to:
- Workflow and failure-mode map. A written map of every system the agent reads from or writes to, such as Salesforce, ServiceNow, Snowflake or DynamoDB, with the failure mode named for each integration. This document later becomes the basis of the runbooks.
- Infrastructure as code. Every Lambda, SQS queue, EventBridge rule, IAM role and KMS key defined in Terraform, CDK or SST and committed to your repository. Nothing should exist in the console that does not also exist in code.
- Evaluation suite. A versioned set of test cases with pass thresholds, run before every release. Our guide to agent evaluation covers how to size and grade one.
- Tool registry and permissions. An explicit list of the tools each agent may call, with scoped credentials per tool. Every write action should carry idempotency keys from day one.
Run
The run phase is where most of the engagement's value builds up, because this is where the agent meets real traffic. It is also the phase most proposals describe most vaguely, so its responsibilities should be listed out:
- Observability. Dashboards for P50, P95 and P99 latency, time-to-first-token, tool-call error rates, token spend per workflow and dead-letter queue depth. The client should have read access to every one of them.
- Release management. Prompt, model and tool changes shipped through staged rollouts with shadow mode and rollback, as described in our post on agent release management. Each release should reference the evaluation run that cleared it.
- Model lifecycle. Model-version pinning, deprecation tracking and migration testing before a vendor retires an endpoint. The model is a dependency with a retirement date, and someone has to own that calendar.
- Cost control. Monthly spend reporting by agent and workflow, with routing and caching changes proposed when the numbers drift. Inference should bill to your AWS account so you can check the numbers yourself.
On-Call
On-call is the line item that separates managed operations from consulting. The provider should name the rotation, the escalation path and the response times by severity — and those terms belong in the contract, where they can be enforced.
Incident handling should follow written playbooks for the failures agents actually produce: retry loops, stale-state reads, partial writes that need a compensating transaction, and prompt-injection attempts. Our agent operator playbooks post outlines what those documents should contain, and the retry and guardrail patterns behind them come from reliability engineering for regulated AI.
Who Builds, Who Runs, Who Gets Paged?
The fastest way to test a proposal is to ask for a responsibility matrix. If the provider cannot fill in every row below, the gaps are where your incidents will land.
| Activity | Managed provider | Client team |
|---|---|---|
| Workflow design and acceptance criteria | Drafts and proposes | Approves; owns business rules |
| Infrastructure as code | Writes and maintains | Owns the repository and the AWS account |
| Evaluation suite | Builds it and runs it on every release | Adds cases; signs off on thresholds |
| Production releases | Runs the staged rollout | Approves changes to high-risk actions |
| First-line on-call | Primary rotation, 24x7 or business hours per contract | Escalation contact for business decisions |
| Model migrations | Tests and schedules the cutover | Approves the cutover |
| Access reviews and audit evidence | Produces CloudTrail and change logs | Reviews quarterly; presents to auditors |
| Spend | Reports and optimizes | Pays AWS directly; sets budget alarms |
Note that the client keeps approval over anything that changes what the agent is allowed to do. That follows the supervised-autonomy model in The Trust Ladder, where an agent gains autonomy only as its measured accuracy earns it.
The provider should own builds, releases, first-line on-call and model migrations. The client should own the AWS account, the repositories, the business rules, and approval of any change that widens what an agent may do.
What SLAs And Handoff Terms Should The Contract Specify?
Agent SLAs need more dimensions than a typical hosting SLA, because an agent can be up and still be wrong. Accordingly, the contract should measure three things separately:
- Availability. Monthly uptime of the agent's entry point, with exclusions for upstream model-endpoint outages spelled out. At 99.9%, that allows roughly 43 minutes of downtime per month; at 99.5%, roughly 3.6 hours.
- Response time by severity. For instance, 15 minutes to acknowledge a Sev-1 (the agent is writing bad data or has stalled on a revenue workflow), one hour for a Sev-2, and next business day for a Sev-3. Acknowledgment times mean little unless a time-to-mitigate is defined alongside them.
- Quality floor. A minimum pass rate on the evaluation suite, measured on every release and on a sample of production traffic each week. Dropping below the floor should trigger an automatic rollback and a written root-cause review.
Handoff terms matter just as much, because every managed engagement eventually ends or changes scope. At minimum, the exit package should include:
- Code and configuration. All infrastructure as code, prompts, tool definitions and evaluation suites. These should already live in your repositories, so the handoff is a permissions change and nothing has to be transferred.
- Runbooks and incident history. Every playbook, plus a log of past incidents, their root causes and the fixes applied. This history is often worth more than the code, because it records the failure modes your systems actually produce.
- Shadow-mode overlap. A 60-to-90-day period in which your team runs the rotation while the provider shadows. Agree the price for this period at signing, while you still have leverage.
- Credential revocation. Removal of every provider role and key on the final day, confirmed against IAM Access Analyzer and CloudTrail. The contract should name who signs off on that confirmation.
Where Should The Data Boundary Sit?
For most regulated enterprises, the answer is simple: the agents run in your AWS account, and the provider's engineers come to the data instead of the data going to them. This is the term least worth compromising on.
- Your account, your keys. Bedrock invocations, vector stores, logs and queues sit in your AWS organization, encrypted with KMS keys you control. Revoking the provider then comes down to removing roles.
- Federated, logged access. Provider engineers assume scoped IAM roles through your identity provider, with session-level CloudTrail logging and no long-lived access keys. Break-glass roles for production writes should require a second approver.
- Private network paths. Model traffic runs over PrivateLink VPC endpoints wherever the service supports them, so prompts and completions do not cross the public internet. Isolation for tool execution follows the same rule, as covered in our post on agent sandboxing.
- Retention you set. Prompt logs, traces and transcripts are kept for the period your policy defines, then deleted. Our guide to agent data retention covers how to set those windows.
For HIPAA-covered workloads, a provider whose engineers can reach PHI is a business associate and needs a signed BAA, and that compliance review should come before any discussion of model quality. The architecture in our post on Bedrock RAG and data sovereignty shows how retrieval stays inside the same boundary.
A managed agent's data should stay inside your own AWS account, encrypted with your KMS keys, and provider engineers should reach it only through scoped, CloudTrail-logged IAM roles you can revoke.
Managed, In-House Or Hybrid: How Do The Models Compare?
No operating model is right for everyone. The choice usually depends on how many agents you plan to run and how stretched your platform engineers already are, and our post on the agent operating model covers the organizational side.
| Criterion | Managed | In-house | Hybrid |
|---|---|---|---|
| Time to first production agent | Weeks, if the provider has run the pattern before | Often a quarter or more, including hiring | Weeks, with the provider leading the build |
| Staffing required | A product owner and an approver for high-risk changes | A sustainable on-call rotation commonly needs four to five engineers | Two to three engineers who own rules, evaluations and approvals |
| 24x7 on-call | Included, if contracted | You build and staff the rotation | Provider is primary; your team is secondary |
| Institutional knowledge | Concentrated at the provider unless runbooks live in your repositories | Stays in-house, but leaves when engineers leave | Shared; strongest when your team owns the evaluations |
| Cost profile | Retainer plus AWS spend at cost | Salaries plus AWS spend | Smaller retainer plus partial headcount |
| Best fit | First one to five agents, thin platform team | Ten or more agents, or agents that are core to the product | Enterprises scaling past a pilot |
Hybrid tends to fit enterprises moving past a pilot: the provider carries the rotation and the model lifecycle, while your team owns evaluation thresholds and approvals. It also makes a later move in-house far less abrupt, because your engineers already understand the system.
Managed fits a first one to five agents with a thin platform team. In-house fits ten or more agents that are core to the product. Hybrid fits enterprises scaling past a pilot that want to build skills internally.
How Do Infrastructure Leaders Evaluate A Managed Agent Provider?
Credentials and case-study logos tell you little about how a team behaves during an incident. Instead, score each provider against criteria you can check against real artifacts:
- Show me a runbook. Ask for a redacted production runbook and a recent incident postmortem. A team that runs agents in production will have both on hand.
- Walk me through a model migration. Ask how they handled their last forced model retirement, including evaluation results before and after the cutover. Our post on model deprecation and migration describes what a disciplined answer sounds like.
- Explain your evaluation method. Ask how they set pass thresholds, who does the grading, and how they control for grader bias. If an LLM judge is involved, ask whether they have measured its position bias and agreement rate.
- Show me the audit trail. Ask for a sample of what an auditor would receive for a single agent decision: inputs, tool calls, model version and approver. Our guide to agent audit trails lists the fields that matter.
- Define idempotency. Ask how they stop a retried write from creating a duplicate record in your CRM. A vague answer here usually means duplicate records later.
- Name the on-call engineers. Ask who is actually on the rotation and where they are based. If the rotation is really an account manager forwarding emails, it will show in your Sev-1 response times.
Each of these questions produces evidence you can check. After all, the provider is asking you to trust them with production systems, and evidence is a reasonable price for that trust.
Ask for artifacts: a redacted runbook, a recent postmortem, evaluation results from the last model migration, and a sample audit record for one agent decision. Teams running agents in production have all four.
Red flags in a managed agent proposal. Any one of these is worth raising before signing:
- Agents hosted in the provider's own AWS account or SaaS tenant, with your data copied in.
- Per-seat or per-credit pricing, with model spend marked up and hidden.
- No evaluation suite in the deliverables, or one that lives only in the provider's repository.
- SLAs that cover uptime but say nothing about response time or quality.
- No written exit terms, or exit terms priced only at the time of exit.
What Do The First 90 Days Look Like?
A well-run engagement follows a predictable sequence, and the first 90 days set the operating rhythm. Here's how that period typically breaks down:
- Weeks 1–2: Discovery. Workflow mapping, access setup through your identity provider, and agreement on the evaluation set and severity definitions.
- Weeks 3–6: Build. Infrastructure as code, the tool registry, the evaluation suite, and the first agent running in shadow mode against real traffic with no write access.
- Weeks 7–10: Supervised production. Writes switched on behind human approval for high-risk actions, with on-call active and dashboards live.
- Weeks 11–13: Steady state. Approval gates relaxed where the evaluation data supports it, the first monthly cost and quality review, and a finished set of runbooks.
Keep in mind that the timeline stretches when integrations are brittle or approvals are slow, and both are common in regulated environments. Be wary of a provider who quotes a fixed timeline without asking about your change-approval process.
Frequently Asked Questions
The questions below cover commercial and transition terms that come up once the architecture is agreed. Each one is worth settling before the statement of work is signed.
How is a managed agent operations engagement usually priced?
Most engagements split into a one-time build fee and a monthly run retainer, with model inference billed to your own AWS account at cost. Tie the retainer to named agents and a defined incident scope, not to seat counts.
Can we take a managed agent in-house later?
Yes, if the contract says so up front. Require infrastructure as code, prompts, evaluation suites and runbooks to live in your repositories from day one, plus a 60-to-90-day exit period with shadow-mode overlap.
What availability SLA is realistic for a production agent?
An agent can only be as available as its model endpoint and integrations, so 99.5% to 99.9% monthly is a common range. At 99.9%, that allows roughly 43 minutes of downtime a month, so define exactly what counts as down.
Who owns the work when a model is deprecated?
The managed provider should own version pinning, migration testing and the cutover, on a timeline tied to the vendor's retirement date. Your team approves the cutover after reviewing the evaluation results.
Does a managed provider need access to our PHI or customer data?
Only through scoped, logged IAM roles inside your account. Engineers should assume roles with session-level CloudTrail logging, never use long-lived keys, and never copy production data into the provider's environment.
Scoping A Managed Agent Engagement
If you're weighing managed, in-house or hybrid operations for your first production agents, the team at iSimplifyMe builds and operates agent systems inside client AWS accounts across CRM, ticketing and data warehouse environments every week. Reach out for a working session — we'll map your workflow, draft the responsibility matrix and SLA terms, and name the failure modes you're about to hit.