llmops-platform-engineering

1# LLMOps Platform Engineering
2 
3Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.
4 
5## When to Use This Skill
6 
7- Building an internal platform for teams to deploy and manage LLM-powered features
8- Designing CI/CD pipelines that include model evaluation gates
9- Setting up A/B testing infrastructure for model versions
10- Creating Kubernetes-based model serving infrastructure
11- Establishing governance workflows for model promotion
12 
13## Prerequisites
14 
15- Kubernetes cluster with GPU node pools (or cloud inference API access)
16- Container registry (Harbor, ECR, GCR, or ACR)
17- CI/CD system (GitHub Actions, GitLab CI, or Argo Workflows)
18- Observability stack (Prometheus + Grafana + OpenTelemetry)
19- Model registry (MLflow or custom metadata store)
20 
21## Outcomes
22 
23- Standardized path from experiment to production
24- Safe model rollout with quality and safety gates
25- Repeatable infra modules for inference, vector DB, and observability
26- Clear ownership model across platform, app, and security teams
27 
28## Reference Architecture
29 
301. **Control Plane**: model registry, prompt/version catalog, policy checks, eval pipeline.
312. **Data Plane**: inference gateway, vector database, cache, feature store.
323. **Ops Plane**: telemetry, alerting, SLO dashboards, cost analytics.
334. **Security Plane**: IAM boundaries, secret rotation, content filters, audit logs.
34 
35## Model Promotion Pipeline
36 
37```yaml
38…