Why the model size decides everything
Installing the app takes two minutes. The number you pick inside it is what makes local AI usable or useless.
The usual first attempt goes like this. You install Ollama, look at the list of models, download the biggest one your disk will hold and ask it a question. It takes ninety seconds to start answering, the fans spin up, and you close it and go back to whatever you were paying for.
The model was simply larger than the memory in your laptop. Your machine spent the whole time shuffling it between RAM and disk instead of running it.
So the first thing to find is your installed RAM. That one number tells you which Qwen size to download, and everything after it is copying a command.
One honest limit before you spend twenty minutes here: a local model will not match a paid assistant on hard reasoning or long research. It is worth having for drafts, summaries, note rewrites and files you would rather not upload.
Pick your Qwen size
- 8GB
- Run ollama run qwen3.5:2b. About 2.7GB to download. If it still drags, drop to qwen3.5:0.8b at about 1GB.
- 16GB
- Run ollama run qwen3.5:4b. About 3.4GB. This is where most laptops should start.
- 24GB
- Run ollama run qwen3.5:9b. About 6.6GB.
- 32GB
- Run ollama run qwen3.5:27b. About 17GB. Expect it to feel noticeably slower than the 9B, and keep the 9B installed as a fallback.
- 64GB or more
- Run ollama run qwen3.5:35b. About 24GB.
- Apple silicon, any size
- Ollama also publishes MLX builds of each size for M-series Macs. Add -mlx to the tag, for example ollama run qwen3.5:4b-mlx. The download is a little larger, about 4GB for the 4B.
Install it and run your first model
Roughly twenty minutes, most of which is the download.
Find your installed RAM
2 minutesOn a Mac, open the Apple menu, choose About This Mac and read both Chip and Memory. On Windows, open Settings, then System, then About, and read Installed RAM. On Linux, run free -h in a terminal and read the total column of the Mem row.
Why this matters
Use the total figure, not the available one. Available memory changes every time you close a browser tab, so a model chosen against it will be too big the moment you reopen your work.
Close anything heavy before the first run. The sizes above assume the model gets most of the machine to itself.
Install Ollama
3 minutesDownload the installer for your operating system and run it. It covers macOS, Windows and Linux, and it installs the command used in the next step.
Note. On a Mac, Ollama needs macOS Sonoma 14 or newer. Intel Macs run it on the CPU only, so it will be slower than an M-series machine and cannot use the MLX builds.
Run the size that matches your RAM
5 to 15 minutesOpen Terminal on macOS or Linux, or PowerShell on Windows, and paste the command from your row above. On a 16GB laptop that is ollama run qwen3.5:4b.
Why this matters
The first run downloads the model, which is where the wait is. Every run after that starts in seconds and needs no connection.
Give it something real and watch the speed
2 minutesType a job you would actually hand over, such as turning a page of notes into three bullet points. If the reply crawls or the machine warns about memory, type /bye and run the next size down.
Why this matters
/bye leaves the chat without removing anything. Both models stay on disk, so switching between them later costs nothing but the typing.
Turn the Wi-Fi off and ask it again
1 minuteConfirm it still answers with no connection. That is the proof the download finished and that nothing is being sent anywhere.
A 4B model that answers instantly is worth more than a 27B model that barely runs.
Use the same model inside VS Code
Continue is a VS Code extension that talks to the Ollama already running on your laptop. Nothing new downloads except the extension and one small autocomplete model.
Install the Continue extension
1 minuteSearch for Continue in the VS Code Extensions panel and install it.
Make sure Ollama is running
1 minuteIf Continue cannot see any local models, open a terminal, run ollama serve and leave it running.
Add your model to Continue's config.yaml
5 minutesOpen Continue's config.yaml and paste the block below, changing the model tag to the one from your RAM row. Then select Local Qwen in Continue's model list.
Add a smaller model for tab autocomplete
5 minutesRun ollama run qwen2.5-coder:1.5b, then keep the autocomplete entry in the block below.
Why this matters
Continue recommends this 1.5B coder model for local autocomplete. Autocomplete has to answer between keystrokes, so pointing it at your main chat model makes typing feel laggy.
Continue configuration
Paste this into Continue's config.yaml. Change qwen3.5:4b to the tag from your RAM row if you are on a different size.
Continue config.yaml for local Qwen
name: Local Qwen
version: 0.0.1
schema: v1
models:
- name: Local Qwen
provider: ollama
model: qwen3.5:4b
roles:
- chat
- edit
- apply
- name: Qwen Coder autocomplete
provider: ollama
model: qwen2.5-coder:1.5b
roles:
- autocompleteThe Apple silicon route
Skip this unless your Mac's chip starts with M. Rapid-MLX is a separate engine that replaces Ollama rather than adding to it, it runs only on Apple silicon, and it uses its own model names. Get Ollama working first, then try this if you want more speed out of the same machine.
Install Rapid-MLX
5 minutesInstall Homebrew first if you do not have it, then run brew install rapid-mlx.
Start the server on the model it recommends
10 minutesRapid-MLX picks by RAM the same way you just did, with its own model names. On 16GB to 17GB it suggests qwen3.5-4b-4bit, on 18GB to 23GB qwen3.5-9b-4bit. Start one with rapid-mlx serve qwen3.5-4b-4bit.
Why this matters
Under 16GB its recommendation is not a Qwen model at all. Run rapid-mlx models and take what it suggests for your machine rather than forcing a size across from the Ollama table.
Point your editor at the local server
2 minutesRun rapid-mlx launch continue-dev. It writes the local address into Continue for you. The same command works with claude-code and cursor in place of continue-dev.
Why this matters
The server listens on http://localhost:8000 and answers on an OpenAI-compatible route, which is why editors built to talk to a cloud endpoint can be pointed at it without any other change.
What offline actually protects
A prompt typed into a model running on your laptop does not leave the machine. That holds only while the model and the route stay local. Ollama also publishes cloud tags, and Rapid-MLX can route to a cloud provider, and anything sent through those reaches a server like any other online tool. Check the tag you are running before you paste something you would not upload.
Where it holds up and where it does not
Local Qwen earns its place on volume work, not on the thinking.
It is good at the jobs where you already know the shape of the answer and want it typed faster: notes into bullets, a long document into a summary, a paragraph rewritten, variables renamed, a first pass at a draft.
It gets worse the further you move from that. Multi-step reasoning, long research, anything needing current information from the web, and anything where a wrong answer is expensive.
Keep the paid assistant for those. What the local model buys you is that the boring half of the work carries on during a flight, on hotel Wi-Fi, or with a file you would rather not hand to anyone.
Confirm it works
- Check that the RAM figure you chose against is the total, not the available memory.
- Run the model once with the Wi-Fi off and confirm it still answers.
- Time a short reply. If it crawls, run the next size down before you build anything on it.
- Confirm the tag you are running does not end in cloud.
- In Continue, confirm the model selector shows Local Qwen rather than a cloud provider.
- Hand it one real job from this week and compare the result against doing it yourself.
Files in this guide
Yours to keep and edit. No attribution needed.