1. Can you run an AI model locally?
Free software such as Ollama and LM Studio now lets anyone run an open-weight AI model on a personal computer, offline and without a monthly subscription. As of November 1, 2026, open-weight models trail the best closed models by about four months on average, according to a May 29 analysis from Epoch AI, and the smaller versions fit on an ordinary laptop.
The short answer to the question is yes, provided the machine has enough memory. Small models need roughly 8GB of video memory (VRAM) on a graphics card, or the same amount of shared memory on a Mac. The largest open models are a different matter, since some are far too big for any consumer device, and that limit shapes everything else in a local setup.
The term open-weight describes a model whose publisher has released the weights, the learned parameter files that contain what the system absorbed during training. Once those files are public, anybody can download them and run the model without asking permission or paying per use. Ollama and LM Studio take care of the technical steps in between. Downloads of Google's Gemma 4 family in the Ollama library, for example, range from about 4.6GB to 20GB, which is within reach of many home machines.
2. Why would anyone run a model on their own machine?
Three motives recur: saving money, protecting privacy and gaining control over how a model behaves. Each addresses a limitation of the cloud chatbots most people use today.
Privacy is the most concrete of the three. A cloud chatbot sends every prompt to servers owned by the provider, so the text of a conversation leaves the user's device. That exposure is not hypothetical: a court order required OpenAI to preserve ChatGPT logs, including conversations users had deleted. A model running locally keeps the exchange on the machine itself. LM Studio's documentation states that the application works entirely offline once the model files have been downloaded, and that documents attached to a chat can also be used without a connection.
Cost is the second argument. Paid chatbot plans commonly run about $20 a month, while the software for running open models is free. The expense does not vanish, however. It moves from a recurring subscription to the one-time cost of capable hardware, which matters for anyone who would have to buy a new computer to get acceptable speed.
Control is the third. Many open models are published under permissive licenses, which allow people to use, modify and redistribute them. DeepSeek's R1 series is released under the MIT license, and its model card says the models allow commercial use and modification. Google's Gemma 4 and OpenAI's gpt-oss are distributed under Apache 2.0. A user who cares about medical questions and has no interest in programming, to take one example, can favor a model tuned in that direction. This kind of customization is mostly possible with models that have been adapted for a particular purpose.
3. What hardware does a local model need?
The deciding factor is memory. Models are measured by parameter count, and the size of the download rises along with it. To run at a usable speed, the whole model has to fit inside the VRAM of a graphics card or inside the unified memory of a Mac, which the processor and graphics unit share.
As a general rule, at least 8GB of VRAM is the entry point for smaller models. People who plan to use these tools seriously may want a gaming graphics card with 24GB or more, a setup with several graphics cards, or a compact machine built around unified memory, such as the Mac Mini or Nvidia's DGX Spark.
Quantization is the technique that makes much of this practical. It stores a model's weights using fewer bits, which cuts memory consumption at the price of some accuracy. Hugging Face documentation illustrates the idea by comparing an int8 representation with a float32 one. A quantized version of a model is therefore smaller and faster, but not quite as precise as the full-size original.
LM Studio publishes its own minimum requirements. On Apple Silicon Macs, it recommends 16GB or more of memory and macOS 14 or newer, although Macs with 8GB can still handle smaller models. On Windows, it asks for at least 16GB of RAM, 4GB of dedicated VRAM and a processor that supports AVX2 instructions. Intel-based Macs are not supported.
Some concrete model sizes help set expectations. Ollama's library lists the 26-billion-parameter Gemma 4 at 16GB to 19GB and the 31-billion-parameter version at 19GB to 20GB. OpenAI says its gpt-oss-20b needs about 16GB of memory, while gpt-oss-120b can run within 80GB.
4. How do Ollama and LM Studio differ?
LM Studio is the more approachable option, built around a polished graphical interface, while Ollama is aimed at developers and experienced users who are comfortable in a terminal. Both are free, and both can be running within minutes.
To set up LM Studio, a user downloads the installer from the company's website, searches for a model inside the application, downloads it and starts chatting. The in-app search draws on Hugging Face, the public hub where open models are shared.
Ollama installs through a short command, a curl script on macOS and Linux or a PowerShell command on Windows, both listed on its download page. Its quickstart then needs only a couple of instructions: the command ollama pull gemma4:e2b downloads a model, ollama run gemma4:e2b opens a chat session, and typing /bye ends it. Ollama also exposes a local API on port 11434 of the user's own computer, so other applications on the same machine can talk to the model.
Ollama additionally offers models tagged as cloud versions, such as gemma4:cloud, which run on Ollama's servers instead of being downloaded. Anyone whose priority is privacy should avoid them, because the prompts leave the machine. They can still be useful for reaching more powerful models than a home computer could host, and API prompts sent that way are described as not being stored or used for training.
5. How can local models be made faster and more useful?
Speed can be improved with a method called speculative decoding. Google's Multi-Token Prediction drafters for Gemma 4 are available on Ollama under the Apache 2.0 license, and they can speed up text generation by as much as three times. A small model proposes several upcoming tokens, and the larger model checks them all in a single pass. During that same pass, the target model contributes one extra token of its own.
Capability can be extended too. Local models can be given web search and file access through servers that follow MCP, the Model Context Protocol. LM Studio added support for MCP in version 0.3.17, which means a locally hosted model can reach tools and data sources that a plain chat window cannot.
6. Where do local models still lose to the cloud?
The four-month gap applies to the most capable open models, and those are the ones that ordinary hardware cannot hold. Alibaba's Qwen3.8-Max, for instance, has 2.4 trillion total parameters, with 95 billion of them active at any moment. Set beside the 80GB within which gpt-oss-120b runs, it is clear that the largest open systems are built for data centers rather than desks. Alibaba's own comparison shows closed models winning 24 of 31 tests.
Memory limits also cause slowdowns well before a model reaches that scale. When a model spills out of VRAM into system RAM, responses become noticeably slower. Longer context windows add to the burden, because they require more memory as well. Gemma 4's small models offer a context window of 128K tokens, and the larger ones reach 256K, so users who feed in long documents need headroom beyond the model file itself.
Cloud services spare users this kind of planning, and frontier cloud models still lead on the hardest tests. For tasks that demand the strongest available reasoning, a hosted system remains the safer choice.
7. Which approach suits which user?
A sensible split is to match the tool to the sensitivity of the task. For private, everyday drafting, a self-hosted local model keeps text on the user's own device. When a task needs fresh information from the web, a service such as Kagi can fill that role, with the understanding that using it involves sending requests through an API.
Developers have a further reason to stay local. Because Ollama serves a model through an address on the same computer, a programmer can build and test an application against it without paying for each request and without sharing proprietary code or customer data with an outside company. That makes local hosting a common choice for prototypes, internal tools and experiments where the data involved is sensitive.
Trade-offs remain on both sides. A local model gives up some raw capability in return for privacy, predictable cost and independence from a provider's pricing or policy changes. A cloud model offers the reverse: stronger results on demand, but with recurring fees and a dependence on someone else's servers. Neither is universally better, and many users end up combining the two.
People weighing the switch should start by checking their available memory, then download one of the smaller models and test it on real work before spending money on new hardware. Because the software costs nothing, the experiment itself carries little risk, and the results show quickly whether a laptop is enough or whether a more demanding machine is justified.