The enterprise guide to local vs. cloud AI architecture

Air-gapped privacy and regulatory compliance

Some data should never transit a network you don't own. Legal teams auditing contracts, healthcare organizations processing PHI under HIPAA and GDPR, finance groups running internal models, and any company analyzing its own proprietary IP all share the same constraint: the value of the analysis is capped by the risk of the transmission.

Local inference deletes that risk rather than managing it. There is no data processing agreement to negotiate with a machine in your own server closet.

• Representative workloads: contract review and clause extraction, PHI de-identification and summarization, internal financial modeling, and RAG over proprietary research that competitors would pay dearly to see.
• The hardware: an M5 Ultra Mac Studio with 512GB of unified memory. At 1.2TB/s of bandwidth it holds a frontier-scale open-weight model and your full enterprise RAG context in one memory pool, on one machine, behind your own badge reader.
• The cloud contrast: no data transmission, no vendor logging, no retention policy to audit, and no clause about model training buried in an updated terms of service. The compliance article in this series covers how this architecture satisfies ISO 27001:2022 and SOC 2 auditors.

The ROI trigger here is not a spreadsheet. It is the first time your general counsel asks where the embeddings live.

High-volume agentic coding and development environments

Agentic coding is the workload that breaks cloud pricing models. An autonomous agent re-reads its context on every turn, runs in loops, and works around the clock. Meter that by the token and the bill curve bends the wrong way. Meter it by the kilowatt and it flattens.

• Representative workloads: local IDE autocomplete, multi-file codebase indexing, automated test-case generation, and recursive CI/CD loops that generate and validate synthetic data all night.
• The hardware: a swarm of four M6 Mac minis on Thunderbolt 5, orchestrated with Exo. Each node serves its own model, so four agents run in parallel without queueing behind one GPU.
• The cloud contrast: zero API rate limits, zero token billing during continuous multi-agent loops, and first-token latency measured on a local bus instead of a WAN. An agent that never waits on a rate limiter is a different product than one that does.

The agentic coding article works through the model selection and the memory bandwidth math, and the M5 Ultra vs. M6 buyer's guide settles which box to buy for it.

Physical AI, robotics, and edge computing

A chatbot can tolerate a 400ms round trip to a data center. A robot arm cannot. When the model output moves metal, the network hop stops being a cost problem and becomes a safety problem, and the only fix is to put the compute on the machine.

• Representative workloads: real-time ROS 2 sensor fusion across LiDAR and spatial cameras, vision-language-action model execution with OpenVLA or Octo, and physical kinematics simulation before a motion plan ever runs on hardware.
• The hardware: an onboard M6 Mac mini. The 2nm process keeps the power draw inside a mobile battery budget, and the Dual 16-core Neural Engine absorbs the perception models while the GPU runs the VLA policy, keeping actuator control latency local and deterministic.
• The cloud contrast: no round-trip latency in the control loop and no internet dependency during autonomous navigation. A robot that stops working when the Wi-Fi does is a demo, not a product.

High-density batch execution and model training

Some jobs don't need to be fast. They need to be relentless. Batch workloads run for twelve hours at full utilization, which is exactly the shape of job that makes per-token pricing look like a rounding error at first and a line item with its own budget review by Q3.

• Representative workloads: overnight log analysis, terabyte-scale document processing, domain-specific synthetic dataset creation, and adversarial red-team evaluation of your own models before an attacker volunteers to do it for you.
• The hardware: multi-node Apple silicon clusters running llama.cpp RPC or MLX swarm topologies. Batch work parallelizes cleanly across nodes because each document is independent, which is the workload swarms were born for.
• The cloud contrast: unlimited continuous execution with no exponential billing curve. The machines cost the same asleep or at full tilt, so the marginal cost of one more overnight run is the electricity.

This is also where fine-tuning lives. Training a domain adapter on your own corpus, on your own hardware, means the training data never leaves the building and the resulting weights are unambiguously yours.

Offline, Field, and SCIF Operations

The last category is the simplest: places where the cloud is not slow or expensive but absent. An AI stack that assumes connectivity is a stack that fails exactly when the environment gets interesting.

• Representative workloads: in-flight development, remote field research, defense work inside a SCIF where radios are surrendered at the door, and disaster response where the network went down with everything else.
• The hardware: a standalone M5 Pro or M5 Max MacBook Pro for the person, and an M6 Mac mini for the site. Both run the same models and the same tooling as the rack back home, so nothing about the workflow changes when the connectivity does.
• The cloud contrast: complete operational independence. There is no degraded mode, because there is no dependency to degrade.

I wrote a meaningful fraction of this series on airplanes with a local model as my pair programmer. The surprise was not that it worked. It was how little I missed the alternative.

The enterprise decision matrix: local Mac vs. cloud

Here is the whole argument in one table. Neither column wins every row, and that is the point: the goal is to route each workload to the side of the table where it belongs.

Read the capacity row carefully, because it is the cloud's strongest and most honest claim. A hosted frontier model will out-reason anything that fits in 512GB. The mistake is paying frontier prices for the 80 to 90 percent of daily traffic that never needed frontier reasoning.

The hybrid deployment framework: local first, cloud fallback

The mature architecture is not local versus cloud. It is local first, cloud fallback, with a router in front deciding which side each request deserves.

In practice that means 80 to 90 percent of traffic, the routine developer queries, the agentic loops, the private RAG lookups, lands on Mac hardware you own at zero marginal cost. The cloud handles the two things it is genuinely better at: context windows beyond what unified memory holds, and the multi-trillion parameter reasoning tasks where the frontier model earns its price.

The pattern holds far above the scale this series is written for. Netflix's AI platform team documented their in-house LLM serving stack, which runs open-weight and custom models on vLLM inside the same scoring service that handles their other production models, with real-time and batch paths side by side. When a company with that much negotiating leverage over API vendors still decides its LLM traffic belongs on infrastructure it operates itself, serving models on hardware you control stops being a small-team economy measure and starts looking like the default.

• Route by sensitivity first. Anything touching regulated or proprietary data stays local, no exceptions. The router enforces the policy so individual engineers never have to remember it.
• Route by capability second. Requests that exceed the local model's context or reasoning budget escalate to the cloud, after the sensitivity gate has already stripped what must not leave.
• Let the fallback earn its keep. The cloud tier also covers hardware failures and demand spikes, which is a better job description than serving your autocomplete.

The setup guide shows the cloud fallback wiring on a real cluster, and the agent hosting article covers running the always-on local tier securely.