llm-cost-optimization

1# LLM Cost Optimization
2 
3Cut LLM costs by 50–90% with the right combination of caching, model selection, prompt optimization, and self-hosting.
4 
5## When to Use This Skill
6 
7Use this skill when:
8- LLM API spend is growing faster than revenue
9- You need to attribute AI costs to teams, products, or customers
10- Implementing caching to avoid redundant LLM calls
11- Deciding when to switch from API providers to self-hosted models
12- Optimizing prompt length without sacrificing quality
13 
14## Cost Levers by Impact
15 
16| Strategy | Typical Savings | Effort |
17|----------|-----------------|--------|
18| Semantic caching | 20–50% | Low |
19| Model right-sizing | 30–70% | Low |
20| Prompt compression | 10–30% | Medium |
21| Provider caching (prompt cache) | 10–25% | Low |
22| Batching offline workloads | 50% (Batch API) | Medium |
23| Self-hosting 7–8B models | 80–95% at scale | High |
24| Quantization | 30–50% VRAM cost | Medium |
25 
26## Track Costs First
27 
28```python
29# Use LiteLLM's cost tracking (automatic per-model pricing)
30import litellm
31 
32response = litellm.completion(
33    model="gpt-4o-mini",
34    messages=[{"role": "user", "content": "Hello"}],
35)
36cost = litellm.completion_cost(response)
37print(f"Cost: ${cost:.6f}")
38 
39# Add custom cost callbacks
40def log_cost(kwargs, completion_response, start_time, end_time):
41    cost = kwargs.get("response_cost", 0)
42    model = kwargs.get("model")
43    user = kwargs.get("user")
44    # Send to your analytics DB
45    db.record_cost(user=user, model=model, cost=cost)
46…