llm-caching

1# LLM Caching
2 
3Cut LLM costs and latency with exact match, semantic, and provider-side caching layers.
4 
5## When to Use This Skill
6 
7Use this skill when:
8- The same or similar queries are asked repeatedly (FAQ bots, support tools)
9- LLM API costs are growing and you need immediate savings
10- Serving high request volumes where repeated queries cause bottlenecks
11- Implementing prompt caching for long system prompts (Anthropic/OpenAI)
12- Building offline-capable AI features that need response persistence
13 
14## Caching Layers
15 
16```
17Request → Exact Cache → Semantic Cache → Provider Cache → LLM API
18             ↓ hit            ↓ hit             ↓ hit
19           instant          ~5ms           50-80% cheaper
20```
21 
22## Layer 1: Exact Match Cache (Redis)
23 
24```python
25import hashlib
26import json
27import redis
28from openai import OpenAI
29 
30r = redis.Redis(host="localhost", port=6379, decode_responses=True)
31client = OpenAI()
32 
33def build_cache_key(model: str, messages: list, temperature: float) -> str:
34    """Deterministic key from request parameters."""
35    payload = json.dumps({
36        "model": model,
37        "messages": messages,
38        "temperature": temperature,
39    }, sort_keys=True)
40    return f"llm:exact:{hashlib.sha256(payload.encode()).hexdigest()}"
41 
42def cached_completion(model: str, messages: list, temperature: float = 0.0,
43                      ttl: int = 3600) -> dict:
44    key = build_cache_key(model, messages, temperature)
45 
46    # Check cache
47    if cached := r.get(key):
48…