Choosing LLM infrastructure means deciding where each part runs and who keeps it working. Managed, hybrid, and on-premises describe useful deployment approaches, but an architecture label alone does not identify the destination of prompts, files, tool results, or operating records.
Draw the model service, agent runtime, data stores, and tool connections separately. Then mark every boundary the task crosses.
Define where the model, runtime, data, and records run before choosing a label.
Compare approaches by responsibility
Scroll horizontally to see all columns.
| Approach | General arrangement | Responsibility to resolve |
|---|---|---|
| Managed | A service operator runs specified components. | Agree which components and operating duties the service covers. |
| Hybrid | Components span local infrastructure and cloud services. | Coordinate identity, networking, data transfers, and incidents across boundaries. |
| On-premises | Specified components run on customer-controlled local infrastructure. | Provide capacity, updates, monitoring, and recovery for those components. |
A managed model service can be called from a customer-operated agent runtime. A local runtime can still send prompts to an external model. Conversely, hosting a model locally does not prove that every tool or operating record remains local.
Specify the scope whenever someone says “on-premises”: does it cover model serving, agent execution, storage, telemetry, or the complete request path?
Start with the data journey
For one representative task, list:
- Input documents and where they are uploaded.
- Context assembled for the model, including tool results.
- Where model inference takes place.
- Which systems the agent may read or change.
- Where conversations, outputs, and operating records are retained.
Review the ordinary route and any alternative route. An approved primary location does not automatically approve a fallback destination. Use the security review guide to turn this map into questions for the relevant owners.
Example: a financial operations pilot
A fictional finance team wants an agent to draft internal explanations of reconciliation exceptions. Its first pilot uses fabricated records and produces drafts for staff review. Before using real records, the team maps the model endpoint, runtime, reference store, and logs.
The team compares three proposals using the same task. For each, it asks who can patch the runtime, restore access to reference data, investigate an incomplete draft, and replace an unavailable model. It also asks whether the required model can be operated with the proposed capacity.
The resulting decision brief identifies the approved data path, component owners, operating effort, and unresolved questions. It makes no claim that selecting a deployment label establishes regulatory compliance. For the product context, see AI agents in financial services.
Include the cost of operating the choice
Compare model usage with the rest of the work: compute capacity, storage, connectivity, monitoring, updates, and staff time. Customer-operated infrastructure requires someone to maintain it; a managed service still requires your team to configure access and assess whether it fits the task.
Test the proposed arrangement under an unavailable dependency and a planned update. Define the person who decides whether the pilot can resume. A deployment that looks simple on a component diagram may still have a difficult incident handoff.
Match the architecture to a documented offer
Ask a supplier to identify the supported components, customer duties, data destinations, and recovery behavior in writing. Keep optional architecture ideas separate from currently offered deployments.
In AgentShelf
The documented offer includes AgentShelf-managed services and supported customer-managed or external runtimes. These are the appropriate starting points for an evaluation. The general hybrid and on-premises comparisons above do not establish a hybrid or full on-premises AgentShelf offering. Confirm the supported arrangement for your requirements before planning a deployment.
Continue with LLM infrastructure fundamentals and workspaces and external agents to distinguish the components and their bindings.