Something is shifting in how the most regulated organisations buy AI, and it is worth naming plainly. A growing set of large institutions in finance, healthcare, and government are pulling AI in-house: running open models on their own infrastructure, behind their own walls, wired into agentic systems they own rather than rent. This is not a security team being difficult. It is a considered response to a constraint that was always there and is only now satisfiable. A bank's transaction records, a hospital's patient files, a government agency's case notes: for these, sending the data to a vendor's endpoint over the internet is not a preference to be weighed against convenience. It is a line the regulation, the contract, or the board will not let anyone cross. The data cannot leave the building, and until recently that meant the good AI could not come in. That is the part that changed.
What changed is that open models became good enough for the operational work, which is most of the work. Not necessarily for the hardest frontier reasoning, but for the tasks that actually fill an institution's day: classify this document, extract these fields from that form, retrieve the right passage out of millions, draft a reply a person will check before it sends. The interesting question was never the benchmark score of the largest hosted model. It was whether a competent model, wrapped in a serious system, holds up in production. And that question we can answer from experience, because the systems that answer it were never really about the model in the first place. This is the quiet reframing under the whole shift. For years the assumption was that the frontier model was the product and everything around it was plumbing, so of course the good AI seemed to live wherever the best model was hosted. Invert that, treat the model as one replaceable part inside a system you own, and the geography of the whole thing changes.
Take a regulated decision system at scale. One of our principals ran fraud decisioning at a bank that makes sixteen million decisions a month, and has for more than five years. What kept it reliable was never a single model call: it was the thresholds, the review lane for the uncertain cases, the audit trail, and the evaluation that caught drift before customers felt it. Every one of those is a property of the system, not the model, and every one of them runs perfectly well on infrastructure the institution owns. A system that has to hold for five years and stay explainable to a regulator is, if anything, a more natural fit for private infrastructure than for a vendor endpoint that can change under it between one week and the next. The properties a regulator cares about are precisely the ones you would rather not rent. Explainability, stability, and a decision you can reconstruct are easier to guarantee on hardware you control than on a service that can be updated, deprecated, or repriced on someone else's schedule.
Retrieval tells the same story from a different angle. One of our principals built a thirty-million-document search index at a national newsroom, with production applications sitting on top of it. An index of that shape is infrastructure, not magic: it runs inside a private network exactly as well as anywhere else, because retrieval quality was always an evaluation and extraction problem, never a question of which hosted API you called. Once you accept that the hard parts of these systems (the evaluation set, the retrieval layer, the thresholds, the human gate) live outside the model, the case for keeping the whole thing inside your own walls stops being a compromise and starts being the obvious architecture. It stops feeling like giving something up. An institution that already runs its own data centres, its own identity, its own audit and backup is not taking on a strange new burden by hosting the model too. It is extending a discipline it already has to one more component, and getting a system it can inspect end to end in return.
The word doing new work in these conversations is 'agentic,' and it deserves a concrete meaning rather than a hopeful one. An agentic system is a model given tools and a loop: it takes a request, retrieves what it needs, calls the model, invokes a tool to actually do something, and repeats until the task is done. Pulled in-house, every step of that loop runs on the institution's own hardware. The agent retrieves from the institution's own index, calls a model hosted in the institution's own environment, and invokes tools that touch the institution's own systems. The consequential step, the one that moves money or changes a record or sends something to a patient, routes to a human gate before it executes. And the audit trail of the whole loop lands on the institution's own storage, not a vendor's. That is what agentic infrastructure behind your own walls actually means, step by step.
We will not pretend private AI is free, because the honest version is what makes it trustworthy. Running your own models means you carry the inference infrastructure, the evaluation harness, and the on-call that a hosted API quietly carried for you. For a team sending a few thousand requests a month, on data that could live anywhere, a hosted key is the right answer and we will say so. Private AI earns its cost in three situations, and mostly only these three: when the data genuinely cannot leave, when the volume makes per-token pricing punitive at scale, and when the model has to be a stable, owned artifact rather than a dependency that shifts underneath a system that must stay explainable. Outside those cases it is over-engineering. Inside them it is the only thing that actually works, and the institutions now building it know exactly which case they are in. The clarity matters more than the conclusion. A team that has honestly worked those three questions and landed on a hosted key has made a good decision; a team that went private to look serious, without the constraint to justify it, has bought cost it will come to resent. The point is to match the architecture to the real constraint, not to the mood of the moment.
Outside those cases it is over-engineering. Inside them it is the only thing that actually works.
None of this is a new philosophy. It is the same thesis we apply everywhere, moved behind the client's own walls: layer AI onto the systems the organisation already runs rather than replacing them, keep a human on the decisions that are expensive to get wrong, and evaluate before anything reaches production. The only thing that moves is where it all lives. The model that makes the call, the human who signs off, the record of what was decided: on the institution's hardware, inside its network, under its control. For finance, healthcare, and government, that is not a diminished version of AI they are settling for. It is the version they can actually deploy, and the reason the demand is moving in-house is simply that, at last, it can.