AI Future Fridays: Local AI and the Machines You Own

Welcome to AI Future Fridays. Every week this column takes one force shaping the near future, tells you the truth about it, and hands you something to build before Monday. This week the force is a quiet one: the model came home.

For three years the arrangement was fixed. You typed a question, your words traveled to a rented building full of someone else's computers, and an answer came back a second later. Convenient, and completely one-sided. You never saw the logs. You couldn't audit the retention policy. When the price changed or the model got quietly swapped, you found out afterward like everybody else.

That arrangement is ending, and not because of a manifesto. It's ending because the math changed.

The Local AI Signal From llama.cpp and Hugging Face

The clearest signal this year wasn't a product launch. It was an acquisition that read like a promise.

Rewind first. On March 10, 2023, a Bulgarian developer named Georgi Gerganov published a small C++ project called llama.cpp and showed something that wasn't supposed to be possible yet: a seven billion parameter language model answering questions on a MacBook, on the CPU, with no graphics card and no connection to anything. People assumed it was a stunt. It was the starting gun.

Three years later, on February 20, 2026, Hugging Face announced that Gerganov and the GGML team were joining the company, with the stated goal of keeping llama.cpp fully open source and community driven while making local inference easy enough for people who are not developers. Read that sentence again. The largest open model platform in the world looked at the landscape and decided the thing worth investing in was the software that runs models on your own desk. The commercial gravity of the last three years pointed entirely the other way, toward rented capacity and per-token billing, which is the world hosted agent runtimes were built for. The signal here is that the gravity flipped.

The Forecast for On-Device AI Through 2027

The near-term read is unusually easy, because this one is already in the manufacturing pipeline and the legislative calendar rather than in somebody's roadmap deck.

Start with the hardware. Counterpoint Research expects genAI-capable phones to hit 45% of global shipments in 2026 and 52% in 2027, with Chinese manufacturers pushing the capability down into mid-range handsets rather than keeping it as a flagship feature. That matters more than the percentage suggests. When the cheap phone runs a model, the assumption that AI is something you rent stops being universal.

Now the demand side, which is less discussed and more interesting. A CNBC and SurveyMonkey workforce survey published in May 2026 found that roughly 65% of workers have at some point avoided using AI at all, citing privacy, accuracy, moral and environmental reasons. That's not a rounding error of holdouts. That's most of the workforce declining a tool because of where it sends their words. Local models answer that objection directly, and they answer it without a policy document.

Policy is moving too. The European Commission's Cloud and AI Development Act, published on June 3, 2026, defines four assurance levels for cloud and AI sovereignty, from "processed inside the Union" up to full transparency and control over the software supply chain with no third-country interference. It also names open source as a resilience strategy in plain text. Whatever you think of the politics, procurement officers across a continent now have a vocabulary for asking where a model runs, and vendors will build to answer it. Anyone who has wired up a security operations stack already knows the shape of that question: the interesting part was never the tool, it was where the data went afterward.

The cheapest way to keep a secret is to never send it anywhere.

The Backstory of Running Software You Own

Every future has a lineage, and this one is a pendulum that has already swung more than once. Computing keeps alternating between machines you own and machines you rent, and the deciding factor has never been ideology. It's whether the work fits on the thing in front of you.

• 1960s: you rented minutes on a mainframe down the road and waited your turn. Nobody owned compute.
• Late 1970s: the personal computer put the whole machine on a desk, and a generation discovered that owning the hardware meant owning the output.
• 2000s: software as a service moved the programs back off your desk and onto rented servers, and it was mostly a good trade.
• 2022 to 2024: generative AI arrived cloud-first out of necessity. The models were too big for consumer hardware, full stop.
• 2025 to 2026: quantization shrank the models, small models got smart, and the chips grew dedicated silicon for them. The pendulum swung back.

The myth worth killing is that local AI is a compromise for paranoid people. Apple shipped the clearest counterargument: its Foundation Models framework hands any developer direct access to the roughly three billion parameter model already sitting on the device, with no cloud dependency and no per-call cost, and in 2026 it added image input so a phone can read a receipt without the receipt going anywhere. That's not a privacy accommodation bolted onto a product. That's a platform deciding the default should be the device. The same instinct runs through the content provenance work we covered last week: the industry is slowly rebuilding the parts of computing where you can verify a claim yourself instead of taking a company's word for it.

What Should Run Locally and What Should Not

Here's where most enthusiasm goes wrong. People install a runner, ask a small model to do something a frontier model struggles with, get a mediocre answer, and conclude local AI isn't ready. The skill isn't installing anything. The skill is sorting.

Sorted that way, the local model stops being a worse version of a chat assistant and becomes a specific instrument for a specific band of work: the repetitive, private middle of your week. It's the same discipline behind a well-built research workflow, where the value comes from routing each step to the right tool rather than from finding one tool that does everything.

The Countermove: Build a Local-First Task Runner

The street finds its own uses for things. In this case the use is boring and powerful: a small program on your machine that does your weekly grunt work and decides, by rule, what's allowed to leave.

Do the unglamorous parts first. They take an hour and they're what make the rest work.

• Pick one runner: Ollama and LM Studio both install like ordinary apps and give you a local model in about ten minutes. llama.cpp underneath both of them if you want the raw thing.
• Pick two models, not ten: one small and fast for daily work, one larger for when quality matters more than speed. Downloading fourteen models is a hobby, not a setup.
• Close the loop: open your runner's settings and turn off usage reporting, crash analytics and automatic updates. A local model behind a chatty app isn't local in the way that counts.

Now the build, and it's a shape you've met before if you've browsed the workflow directory: a scheduled job that reads a folder and writes one file. A local-first task runner wakes up on a schedule, reads a folder you point it at, and processes each new file. Before it touches anything it applies your routing rule: does this file contain names, contract terms, figures, personal details? If yes, the local model handles it and nothing leaves the disk. If no, it can go to a cloud model for a better answer. Everything it does gets written to a log with the model name attached, so you can audit your own boundary instead of trusting it. That log is the part people skip and the part that turns a setup into a system.

Point it at whatever pile you're behind on. Meeting notes, invoices, a folder of half-finished drafts, the inbox of scanned documents you keep meaning to file. The output doesn't have to be clever. One file, updated daily, listing what came in and what needs a decision from you beats a clever thing you check twice and abandon. A team can point the same runner at a shared docs folder and get a private summarizer out of it, with no vendor review and no data agreement to negotiate.

The Horizon for Digital Sovereignty

Be careful with the word sovereignty. It's a good word and a dangerous one, because the same argument that says "our data should stay on our machines" gets picked up six months later by people who want walls, watchlists and a register of who ran what. That future is not on offer here. No cages, no monitoring regimes, no safety bought with somebody else's freedom. The point of owning your own compute is to reduce the number of people who can compel you, not to increase the number of people you can compel.

And a laptop that answers your questions privately has not fixed any of the things that matter most. Wars are still running. Displacement is still rising. The sky is still warming, and the honest note on energy is that local inference on a small model is cheap while the world builds out data centers at the same time, so this is a shift that stops making the problem worse one errand at a time rather than a shift that solves it. Nobody should oversell a summarizer as a climate policy.

What it does buy is room. A librarian in a town where the broadband drops every afternoon can index twenty years of local records without asking anyone's permission or waiting for the connection to come back. A shop owner can read her own supplier contracts without handing them to a company she's never met. A journalist can work through documents that would put a source at risk if they crossed a network. None of those people are waiting for a vendor to build a feature for them, and none of them have to explain themselves to get it. That's the shape of the thing worth wanting: not a smarter machine, a machine that's yours.

The pendulum swung back this year. Grab it while it's on your side of the arc.

Before Monday: Your Local AI Setup Checklist

You've got a weekend and a computer that's more capable than you've been treating it. Five moves, and the fourth one grows later into a private knowledge base workflow if it earns its keep:

1. Install one runner (Ollama or LM Studio) and pull one small model. Ask it three real questions from your actual week, not test questions.
2. Open the runner's settings and switch off usage reporting and analytics. Write down what you turned off.
3. Write your routing rule in one sentence and stick it somewhere you'll see it. "If I wouldn't paste it into a public forum, it runs locally" is a fine rule to start with.
4. Pick the single folder you're most behind on and point a scheduled summarizer at it. One folder. Resist the urge to do all of them.
5. Check the log on Monday morning and see which files went where. If the routing surprised you, fix the rule before you add anything else.

Frequently Asked Questions

• Do I need an expensive graphics card to run a model on my own computer?+
  No. Any machine bought in the last four years with 16GB of memory will run a small model comfortably, and phones ship with the silicon for it now. A graphics card makes bigger models faster, but the 3 to 8 billion parameter models that handle summarizing, tagging, extracting and rewriting run fine on ordinary laptop hardware. Start with the smallest model that does your job and only move up when it visibly fails, which is the same sizing logic behind any decent data entry workflow.
• Is a local model as good as ChatGPT, Claude or Gemini?+
  Not for hard reasoning, long research or very large context windows. The frontier cloud models are still ahead there and will stay ahead for a while. For the repetitive middle of your week, which is most of it, a small local model is close enough that you won't notice the difference, and it costs nothing per run. Match the model to the job instead of sending every job to the biggest thing available.
• How do I decide what to keep local and what to send to the cloud?+
  Use one rule: if you wouldn't be comfortable pasting the text into a public forum, it runs locally. Client names, contracts, payroll figures, unreleased work and anything under a confidentiality agreement stay on your disk. Public pages and your own throwaway drafts can go anywhere. Write the rule once and let the workflow enforce it, because deciding file by file is exactly where people slip.
• Does running a model locally mean nothing leaves my computer?+
  The inference doesn't leave, but the app around it might. Turn off usage reporting, crash analytics and automatic model updates if you want a clean boundary, and remember that downloading the model is itself a network event. After that it's a closed loop. This is one of the places where knowing what your tools can and can't do saves you from a false sense of security.