The phrase "data center" now conjures a specific image: a windowless building the size of several football fields, a substation beside it, cooling towers behind it, and a hundred-megawatt power contract underneath. The assumption that follows is that this is what AI infrastructure looks like, and that anything smaller is a hobby.
The assumption is worth examining, because it is doing a lot of work. It shapes where capacity gets built, who can build it, which communities get to host it, and what it costs. And for the workload that actually touches users, inference, it is mostly wrong.
This article is about what a small inference installation can and cannot do, drawn from Cobble's experience running one network across three sites, and about where small facilities fit alongside the large ones rather than instead of them.
What "small" means
A small inference installation, in the sense used here, draws tens to a few hundred kilowatts. It occupies a room, a shipping container, a bay in an industrial building, or a small purpose-built structure. It holds a handful to a few dozen racks. It can be sited where electrical service of that size already exists, which describes a great many places: former manufacturing sites, utility buildings, campus facilities, and commercial spaces that once held something else.
That scale is not arbitrary. It is roughly the size at which a facility can be powered from local sources, cooled without industrial water consumption, staffed by a small team, and expanded in increments that a regional organization can finance. Above it, the facility starts to need the things that make hyperscale sites hard to site and slow to build: dedicated substations, new transmission, large-scale water, and years of permitting.
Modular expansion
The most underrated advantage of a small installation is that it grows in small steps.
A hyperscale facility is planned years ahead, built in one or two phases, and filled with hardware chosen before the models it will serve exist. A small installation adds a rack when demand justifies a rack. Cobble's fleet is organized into pools by model architecture, with each pool sized and load-tested for the models it serves. Growing it means adding capacity to the pool that is short, with whatever hardware does that job best at the time, rather than committing to a single generation across an entire building.
This matters because inference demand is lumpy and model architectures change. A facility that can add a pool of accelerators suited to a new sparse model, or retire a pool whose models have fallen out of use, is making decisions at the scale the decisions actually happen. A facility that has to justify a hundred-megawatt expansion cannot.
It also lowers the cost of being wrong. A rack that turns out to be the wrong choice is a small mistake. A building full of them is not.
Localized energy
A facility drawing hundreds of kilowatts can be matched to local generation in a way a facility drawing hundreds of megawatts never can. A rooftop or ground-mounted solar array of modest size supplies a meaningful share of a small installation's load. On-site solar already supplies a measured share of completion power on the Cobble Network, and that share grows with each phase of the installation.
Other local sources follow the same logic. Industrial sites often have electrical service sized for equipment that is no longer there. Municipal utilities sometimes have generation or contracted supply that exceeds demand. Small hydro, landfill gas, and regional wind can each cover a facility of this size without new transmission. The common thread is that the facility fits the energy that is already there, instead of requiring the grid to be rebuilt around it.
There is a second-order benefit. Inference demand is flexible in a way that most industrial load is not. Batch workloads can be scheduled for the hours when local generation is highest, and the fleet's metering already records enough per request to make energy-aware scheduling possible. A small facility can be a good neighbor to its grid rather than a strain on it.
Heat
Every watt that goes into an accelerator comes out as heat, and dealing with it is the central physical problem of any data center. Hyperscale facilities solve it at scale, and often with evaporative cooling that consumes large quantities of water.
A small facility has different options. At tens to hundreds of kilowatts, air cooling with well-designed airflow is sufficient for most of the year in most climates, and the Cobble Network uses no evaporative cooling anywhere. The hardware itself was built for dense, sustained operation in laboratory environments, and runs within its thermal envelope under careful monitoring: temperature and power are tracked per node alongside utilization, and a hot node is treated as seriously as an error spike.
The more interesting possibility is that the heat is useful. A few hundred kilowatts of waste heat is roughly what it takes to warm a large building, a greenhouse, a pool, or a small district heating loop. A hyperscale facility produces far too much heat to use locally and too little temperature to move far. A small facility produces about the right amount for a neighbor to want it. This is not yet common practice, but it is the kind of integration that only becomes possible at small scale.
Regional deployment
A small facility can be placed where its users are. For an application that makes dozens of sequential model calls per task, as agentic systems increasingly do, the difference between a facility in the same region and one across the country compounds on every call. For an organization whose data cannot leave a jurisdiction, a facility inside that jurisdiction is not a nicety but a requirement.
Regional deployment also changes the failure picture. The Cobble Network routes across three separate sites, and the relay steers around whichever one is unhealthy or full. Each additional site is a region that keeps working when another does not. A hyperscale facility is a single very large point of failure, and the outages that take down a cloud region prove it regularly.
The limitations, stated plainly
Small installations are not the answer to everything, and pretending otherwise would undermine the case for them.
They cannot train frontier models. That work requires tens of thousands of accelerators in one place with a network built for it, and no number of small facilities adds up to that. They have less headroom for sudden, very large bursts; a small facility that is full is full, which is why Cobble's relay refuses cleanly with an explicit reason rather than degrading everyone. They carry a higher operational cost per rack than a facility that amortizes its staff across a hundred thousand of them. They depend on the quality of local power and connectivity, which varies. And they require the serving software to be good, because a small facility has no room to waste hardware on inefficiency.
That last limitation is the one Cobble has spent the most effort on. The relay, the pooling by architecture, the queueing, the caching, and the measured energy accounting exist because a small facility has to extract everything the hardware can give. A completion on the network uses roughly a third of the energy of a typical industry equivalent, on reclaimed hardware, because there was no alternative to engineering it that way.
Complement, not replacement
The right picture is not small versus large. It is a division of labor.
Hyperscale facilities will train the models, and they will serve the workloads that genuinely need enormous elastic capacity on demand. Small, regional installations will serve the workloads where predictable capacity, data locality, and operational independence matter more: the hospital system, the municipal government, the regional business, the developer who wants to know exactly what they are getting and who is responsible for it.
That second category is larger than the industry's current geography suggests, and it is growing. The communities in Kentucky with whom Cobble is planning municipal installations are not trying to compete with hyperscale. They are trying to own the part of AI that serves them, at a scale that fits where they are.
Not every workload needs a football field. Most of the ones that reach a person do not.
The Cobble Team