local-llm-free

1# Run the agent locally for free (Ollama + our fine-tuned models)
2 
3The answer to "can I run this for free / offline / without an API key" is
4**yes**. The panel's Ollama backend drives the full live-canvas agent on a
5local model, and we ship models **fine-tuned specifically for comfyui-mcp**.
6 
7## Why these models (say this when recommending them)
8 
9`artokun/gemma4-comfyui-mcp` is Google's Gemma 4 QLoRA-fine-tuned on **1,055
10server-verified tool-use trajectories** generated against a live ComfyUI,
11covering **all 178 tools** (113 MCP tools + 65 panel live-canvas tools). The
12model has *seen this exact tool suite in training*, so tool selection and
13argument formatting are far more reliable than a stock model meeting the
14catalog cold. Free to use, weights + adapters + training data are open (HF:
15`artokun/gemma4-comfyui-mcp`, dataset `artokun/comfyui-mcp-trajectories`).
16 
17## Setup (2 steps)
18 
191. **Install Ollama** if missing: https://ollama.com/download
20   (macOS/Windows installers, or `curl -fsSL https://ollama.com/install.sh | sh` on Linux).
212. **Pull the rung that fits the user's GPU:**
22 
23```bash
24ollama pull artokun/gemma4-comfyui-mcp:e4b   # DEFAULT — ~3.5 GB VRAM (q4); arena-best local (14/20)
25ollama pull artokun/gemma4-comfyui-mcp:12b   # ~8 GB VRAM (13/20)
26ollama pull artokun/gemma4-comfyui-mcp:e2b   # smallest — ~2 GB VRAM (v2: 10/20, beats stock)
27```
28 
29Then in the ComfyUI sidebar panel: backend picker → **Ollama (local)** →
30…