My goal with this little project was to explore the capabilities and performance of local LLMs for AI assisted development. My test machine is an M4 Max MacBook Pro with 128 GB RAM. Initial testing showed LM Studio was faster than Ollama with CLI agentic coding tools. Initial testing also showed MLX models were faster than GGUF models.
My Claude Code configuration is based on the Trail of Bits configuration, so there is a large ~/.claude/CLAUDE.md file, many skills, etc.
# introduction
This is a fresh new project that needs to be created from scratch.
It is a pure HTML and JavaScript web application.
The purpose of this web app is to help new first time parents track newborn baby
feeding, sleeping, and diaper changing.
# design and behavior
The page should have separate buttons at the top for feed, sleep, and diaper.
The buttons are oriented horizontally next to each other.
Each button logs the current time and date to a log table at the bottom of the page.
The log table has the most recent entries listed first at the top.
The columns in the log table are date, time, and category (feed, sleep, diaper).
There should be elapsed time displays under the three buttons, to display the hours and
minutes since the last feed, sleep, and diaper.
There should also be a reset button that clears the log and starts fresh.
The data for the log table is stored in the browser.
This allows someone to exit the page tab and return to the page later without losing any data.
Data is never sent to a server.
# implementation
1. plan the implementation
2. implement it
3. test whatever can be tested automatically
4. document the project in a README.md file
| Model | Quantization | Size (GB) | Tool | Time (m:s) | Notes |
|---|---|---|---|---|---|
| Devstral Small 2 24B | 4bit | 14.1 | Claude Code | 16:00 | worked great |
| Qwen3.6 35B A3B | 8bit | 37.7 | Claude Code | 4:30 | works with wrong log sorting |
| Qwen3 Coder Next | 8bit | 84.7 | Claude Code | 5:00 | works, but timers not refreshing automatically, needed another iteration |
| Devstral 2 123B | 4bit | 70.3 | Mistral Vibe | 30:00 | crashed OOM during test and documentation, but otherwise worked great |
| Devstral 2 123B | 4bit | 70.3 | LM Studio | 7:20 | needs iteration on a couple fixes |
| Qwen3 Coder Next | 4bit | 44.9 | Claude Code | 3:42 | nailed it (2nd favorite design) |
| Qwen3 Coder Next | 4bit | 44.9 | LM Studio | 0:53 | nailed it |
| GPT OSS 120B | MXFP4 | 65.3 | LM Studio | 0:53 | needs iteration and some reload fixes |
| Claude Opus 4.8 High | ? | ? | Claude Code | 1:23 | nailed it (1st favorite design) |
| Claude Sonnet 4.6 High | ? | ? | Claude Code | 1:12 | nailed it (tied 1st favorite design) |
Large dense models like Devstral 2 are slow and they eat lots of RAM for context. That model documentation says the full size one is intended to run on 4x H100s, which have a lot more horsepower and a lot more memory bandwidth than my laptop. I did manage to get a good result from it though. I might reserve this model for thinking and reasoning tasks instead of agentic coding. This did not work well with Claude Code's large prompts/contexts, chatty nature, and short timeouts. It worked better with Mistral Vibe, but the model was eventually killed for eating too much RAM.
GPT OSS 120B was a lot faster than Devstral 2, but it did not work well with Claude Code. Similar to Devstral 2, I'll likely reserve this for thinking and reasoning tasks.
All of the other models worked well with Claude Code. Performance and quality varied quite a bit.
For the finished product, I used a combination of Claude Opus 4.8 and Qwen3 Coder Next. There are a variety of strategies for combining cloud models and local models for various tasks that I plan to expore more. There are use cases for using local models even when security and privacy are not the primary concerns.