How to Install Your Own LLM on a Mac or PC (Private AI Chat and Image Generation)

Published July 14, 2026 by Aldo Giammusso

You can run a capable AI model entirely on your own computer: private, offline, no subscription, and nothing leaving your desk.

Most people assume AI means a website and a monthly bill. It does not have to. The software that runs a real language model on your own Mac or Windows PC is free, it is mature, and getting to a working chat window takes about twenty minutes.

There is one honest catch, and it is not the software. It is memory. Local AI needs a lot of it, and mid 2026 is an unusually expensive moment to buy it. That gets its own section at the end.

Why Run an LLM Locally?

Privacy is the headline reason. When you type into a cloud chatbot, your prompt leaves your machine, and so does every document you attach to it. If you handle client records, medical notes, or contracts, a local model removes the question entirely. The weights sit on your drive and the work happens on your chip.

The rest of the case is practical:

• No per seat subscription. You buy the hardware once and the software is free.
• It works offline. No connection, no outage, no service going down mid task.
• No rate limits and no usage meter running while you think.
• Full control. You pick the model, keep the version you like, and nobody changes its behavior overnight.

The honest counterpoint: the biggest frontier models in the cloud are still better than anything that fits on a desk. Running locally is about privacy, control, and cost, not about beating the best model in the world.

What You Can Actually Run

The rule that governs everything else: the model has to fit in memory, with room left over. Every decision below, including which machine to buy, comes back to that.

On an Apple Silicon Mac, CPU and GPU share one pool of unified memory, which is why Macs punch above their weight here. But macOS does not hand all of that pool to the GPU. Metal reports a recommended working set that is a large fraction of the total, not all of it. The exact fraction is not a fixed number: it depends on how much memory the machine has, and the sources that document it disagree about where a 48GB Mac lands, putting the share somewhere between two thirds and three quarters. So budget on roughly 32GB to 36GB for the model on a 48GB Mac. Nobody has published a direct measurement at that size, so treat it as an estimate and check your own machine.

Quantization takes one sentence: most local models ship compressed, so a Q4 build stores each weight in about four bits instead of sixteen, making the file far smaller and faster to read at a slight cost in quality.

Put those together and 48GB is the realistic entry point, running a 30B class model comfortably with headroom for context. Qwen3.6-27B, a dense model from April 2026, is about 16.8GB at Q4_K_M and 22.5GB at Q6_K. Qwen3.6-35B-A3B is a mixture of experts model with 35B total parameters but only about 3B active per token, and its 4 bit MLX build is about 21.6GB. Both are Apache 2.0. Gemma 4 offers a 31B dense version, roughly 20GB at 4 bit, and a 26B mixture of experts version. Bigger models are reachable from 48GB only at heavier quantization, where quality starts to slide.

So 48GB is better spent on a 27B dense model at near lossless 8 bit than on a 70B squeezed down to a shadow of itself. Llama 3.3 70B at Q4_K_M is 42.52GB, which does not realistically fit 48GB; at 64GB it becomes comfortable. At 128GB you get roughly 96GB, where 120B class mixture of experts models become a daily driver, since a 4 bit Qwen3.5-122B-A10B is about 69.6GB. Leave a few gigabytes spare in every case: the key value cache grows with how much text you feed the model and sits on top of the weights.

How to Install a Local LLM on a Mac (Step by Step)

Start to finish, this is about twenty minutes on a machine with the memory for it.

Step 1: Check How Much Memory Your Mac Has

Click the Apple menu, then About This Mac, and read the Memory line. That number is the ceiling for everything that follows: 48GB of unified memory is the realistic entry point for genuinely useful models, and 16GB is LM Studio's recommended minimum for small ones. The machine must be Apple Silicon, since Intel Macs are not supported, and both apps require macOS 14 or newer.

Step 2: Install LM Studio or Ollama

LM Studio, currently 0.4.19, is the easiest starting point for anyone who is not a developer: a real Mac app with a built in model browser. Install it from the DMG at lmstudio.ai, with brew install --cask lm-studio, or with curl -fsSL https://lmstudio.ai/install.sh | bash. Ollama, currently v0.32.0, ships a desktop app of its own now: take the DMG from ollama.com/download/mac, or use brew install --cask ollama-app for the app and brew install ollama for the command line server.

Step 3: Download Your First Model

In LM Studio, open the model browser, search, and download. Pick a Q4 build whose file size leaves several gigabytes of headroom under your memory limit. On a 48GB Mac, start with Qwen3.6-27B (about 16.8GB at Q4_K_M, 22.5GB at Q6_K) or Qwen3.6-35B-A3B, a mixture of experts model whose 4 bit MLX build is about 21.6GB but activates only about 3B parameters per token. Both are Apache 2.0. Gemma 4 31B at 4 bit lands around 20GB.

Step 4: Run the Model Locally

In LM Studio, load the model and start typing. With Ollama there is a trap: version 0.32.0 turned the bare ollama command into an interactive agent that defaults to a cloud model, which quietly defeats the whole point of running AI on your own machine. Always name the model, because ollama run <model-name> is what keeps the work local. Then unplug the network and confirm it still answers.

Step 5: Install Image Generation Software

On a Mac, Draw Things is easiest by a wide margin: a native app with no Python and no virtual environments, and on device generation is free, since the paid Draw Things+ tier buys cloud compute you do not need. ComfyUI is the power user path and the reference implementation new models target first, but on Apple Silicon it wants the PyTorch nightly build, and its one click desktop app lives in the Comfy-Org/Comfy-Desktop repository, not the older archived one that still ranks well in search.

Step 6: Download an Image Model

Draw Things has a catalog built in, and you can import anything else from Settings, then the Model dropdown, then Manage, then Import Model, then Select from Files, with the app in Local server mode. Hugging Face and CivitAI are the model sources worth using. Z-Image-Turbo (6B, Apache 2.0) is a light, capable first pick; FLUX.2-klein-4B is the practical FLUX.2 build at Apache 2.0 and about 13GB; SDXL base 1.0 still has the deepest library of LoRAs and ControlNets. One warning: those are weight file sizes, and the text encoder, VAE, and working memory sit on top.

Step 7: Keep Everything Updated

LM Studio and Ollama update themselves, and ComfyUI ships roughly weekly, so check its releases page rather than trusting the version you installed months ago. Before installing anything new, confirm it is alive: GPT4All still carries about 77,000 stars and ranks high in search, but its last release was February 2025 and its last real code change was May 2025. Stars measure history, not maintenance, so check the date of the last release.

How to Do the Same on a Windows PC

The software is nearly identical. The hardware thinking is completely different, and this is where most people waste money.

VRAM Is the Number That Matters, Not System RAM

Generating each token requires reading the entire set of model weights out of memory. Every token. So speed tracks memory read bandwidth, not processing power. A GPU reads its VRAM at hundreds of gigabytes per second, up to about 1,792 GB/s on an RTX 5090, while dual channel DDR5 is roughly an order of magnitude slower. On a PC, the question is never how much RAM the machine has. It is how much VRAM is on the card.

What Fits in 16GB, 24GB, and 32GB

At Q4, weights work out to roughly 0.6 bytes per parameter, with context memory on top. A 16GB card runs 7B to 14B models entirely on the GPU at Q4; 24B gets tight and 32B does not fit. Move to 24GB, as on an RTX 4090, and 32B fits. At 32GB, the RTX 5090 and the top of the consumer market, a 70B model at Q4 is still out of reach, since it needs well over 40GB.

Capacity is only half the story. The 16GB cards are not equivalent: an RTX 5060 Ti 16GB reads memory at 448 GB/s, an RTX 5070 Ti at 896 GB/s, an RTX 5080 at 960 GB/s. The same model fits on all three and runs at very different speeds.

Offloading Works, But It Falls Off a Cliff

A model bigger than your VRAM still loads, because Ollama and LM Studio spill the extra layers into system RAM. It just gets slow, since any layer on the CPU stalls the GPU every decoding step. The penalty is a cliff, not a gradient: nudging the offload slider a few layers buys very little, and getting the whole model onto the card is worth more than every partial step combined. If a model does not fit, change the model or the quantization instead of splitting the difference.

64GB of System RAM Is Headroom, Not a VRAM Substitute

This is the misunderstanding that costs people the most money. System RAM is for the operating system, file caching, and loading models off disk. It is not an inference tier and it does not extend your GPU. Buying 64GB of DDR5 instead of a bigger card is a fast way to spend real money and get a slow model. A fast SSD is close to required though, since models run from 5GB to 50GB or more each.

The Software Is the Same

Ollama runs on Windows 10 22H2 or newer, installs from OllamaSetup.exe or with irm https://ollama.com/install.ps1 | iex, needs no administrator rights, and bundles its own CUDA runtime. LM Studio needs an x64 chip with AVX2, and recommends at least 4GB of dedicated VRAM for GPU acceleration. Neither requires the CUDA Toolkit. ComfyUI ships a Windows portable with Python 3.13 and PyTorch CUDA 13.0, plus an alternate on CUDA 12.6 and Python 3.12 for older GPUs, and its desktop app is the easiest start.

What Hardware to Buy (and Why This Is an Awkward Moment)

Here is the part most guides skip: this is a bad moment to buy memory, and memory is the only thing that matters for local AI.

There is a global shortage. AI data centers are projected to consume more than 70 percent of the high end memory chips made in 2026, and manufacturers moved capacity to meet that demand. Everything memory heavy got scarce and expensive. Apple's Tim Cook told the Wall Street Journal he had never seen anything like it in more than 40 years, and called the rising memory and storage costs unsustainable.

On the NVIDIA Side

Every RTX 50 card with 16GB or more currently sells above its MSRP. These are the launch MSRPs, and in July 2026 they are a reference point rather than a price you will pay:

• RTX 5060 Ti 16GB, $429 MSRP, 448 GB/s. The cheapest way into 16GB, and the slowest of them.
• RTX 5070 Ti, $749 MSRP, 16GB at 896 GB/s. The best balance of capacity and bandwidth.
• RTX 5080, $999 MSRP, 16GB at 960 GB/s. Fastest memory of the 16GB cards.
• RTX 5090, $1,999 MSRP, 32GB at 1,792 GB/s. The consumer ceiling for capacity and speed both.
• 64GB DDR5-6000 kit, roughly $800 as of July 2026 and moving week to week. Headroom, not a substitute for VRAM.

As an Amazon Associate we earn from qualifying purchases.

NVIDIA also cut RTX 50 supply to its board partners by roughly 15 to 20 percent, and reporting indicates the 16GB cards took the sharpest cuts while 8GB cards got higher allocation to keep shelves stocked. The shortage lands hardest on exactly the cards local AI buyers need, and manufacturers have confirmed that supply constraints are affecting production and restocking. Do not wait for the rumored higher VRAM Super refresh either: it has been delayed indefinitely, for the same reason, since it depends on scarce 3GB GDDR7 modules.

On the Apple Side

Apple raised Mac prices on June 25, 2026, and said why in plain language: the rapid expansion of AI data centers has created an extraordinary surge in demand for memory and storage. Apple also cut its high memory configurations. The Mac Studio is now configurable at 36GB, 64GB, or 96GB only, and the M3 Ultra, the most expensive chip Apple sells, tops out at 96GB. The Mac mini offers 16GB or 24GB with M4, and its M4 Pro version maxes out at 48GB.

Two facts to have straight before you shop. The M5 generation is laptop only: it lives in the MacBook Air and MacBook Pro, and no Apple desktop has an M5 chip, since the Mac Studio runs M4 Max or M3 Ultra. And the laptop now out specs every Apple desktop on capacity, because M5 Max supports up to 128GB at up to 614 GB/s while M5 Pro supports up to 64GB at up to 307 GB/s. Bandwidth does not follow the model years either: M3 Ultra still leads at 819 GB/s, ahead of M5 Max, then M4 Max at 546 GB/s on its 16 core CPU and 40 core GPU tier (410 GB/s at base).

Supply is tight here too: Mac Studio delivery estimates have run 9 to 10 weeks, both it and the Mac mini have gone out of stock, and Cook said the two may take several months to reach supply demand balance.

• Mac mini (M4 Pro), which maxes out at 48GB, is the cheapest Apple route to the entry tier.
• Mac Studio reaches 64GB on the M4 Max 16 core CPU and 40 core GPU tier, or 96GB on the M3 Ultra.
• MacBook Pro (M5 Max) is the only Apple machine that goes to 128GB.

Configuration pricing moves, so price the exact memory tier you want on Apple's own site rather than trusting any figure in a guide, including this one.

What to Actually Do

Buy for the memory. The chip generation matters far less than the number next to unified memory or VRAM, and on a Mac that is the one spec you cannot upgrade later. Expect to pay above MSRP or to wait, and treat the used market as a serious option rather than a fallback. Do not build a plan around prices falling soon: the shortage is projected to run into roughly Q4 2027. If you would rather not sort through this alone, that is what our computer purchase consulting is for.

The Bottom Line

Local AI crossed from hobby project to genuinely usable. The software is free and well maintained, and installing it is a twenty minute job that does not need a developer. A 27B or 35B class model on your own machine will draft, summarize, rewrite, and help with code, with your data never leaving the building.

The cost is not the software. It is memory, and this is a rough moment to buy it. So the honest move is the boring one: buy for the memory, buy what you need rather than what you might need, and be patient. Get the machine right and the AI part is free forever. If a machine on your network is already misbehaving, handle that first, with virus and malware removal or a look from our computer repair shop.

Frequently Asked Questions About Running a Local LLM

How much RAM do I need to run an LLM locally?

On a Mac, 48GB of unified memory is the realistic entry point, because macOS hands the GPU a large fraction of the total rather than all of it, leaving roughly 32GB to 36GB for the model on a 48GB machine depending on the machine. That is enough for a 27B to 35B class model at a sensible quantization. On a Windows PC the number that matters is VRAM, not system RAM: 16GB runs 7B to 14B models entirely on the card at Q4, 24GB reaches 32B, and 32GB is the consumer ceiling.

Is a local LLM as good as ChatGPT?

Not quite, and it is worth being honest about that. The biggest frontier models still beat anything you can fit on a desk, so if you need the strongest possible reasoning, the cloud wins. What you get instead is privacy, no subscription, offline operation, no rate limits, and a model that does not change under you. For drafting, summarizing, rewriting, and coding help, a good 27B to 35B class model is genuinely useful.

Can I generate images locally too?

Yes, on the same hardware. Draw Things is the simplest route on a Mac, and ComfyUI is the power option on either platform. Z-Image-Turbo (6B, Apache 2.0) and FLUX.2-klein-4B (Apache 2.0, about 13GB) are both practical on consumer hardware, and SDXL base 1.0 has the widest LoRA and ControlNet ecosystem. Remember that published figures are weight file sizes, so the text encoder, VAE, and working memory need room on top.

Do I need an NVIDIA GPU, or will a Mac work?

Both work well. A Mac shares one pool of unified memory between CPU and GPU, so a fairly ordinary looking machine can hold a large model, and Apple's MLX framework runs underneath Ollama and LM Studio without you installing anything extra. NVIDIA gives you more memory bandwidth per card and the most mature tooling, but consumer VRAM stops at 32GB. Apple wins on capacity per dollar, NVIDIA wins on speed per card.

Is running AI locally actually private?

Yes, with one caveat. The weights sit on your drive, the math happens on your chip, and nothing needs a network connection, so you can unplug and keep working. The caveat is that some tools reach out by default: Ollama v0.32.0 turned the bare ollama command into an agent that defaults to a cloud model, so run ollama run <model-name> explicitly to stay local. Confirm any cloud compute option in your image app is switched off, then test by disconnecting the network.