observability-and-instrumentation

1# Observability and Instrumentation
2 
3## Overview
4 
5Code you can't observe is code you can't operate. Observability is the ability to answer "what is the system doing and why?" from the outside, using the telemetry the code emits. Instrumentation is not a post-launch add-on — it's written alongside the feature, the same way tests are. If a feature ships without telemetry, the first user-reported bug becomes archaeology instead of a query.
6 
7## When to Use
8 
9- Building any feature that will run in production
10- Adding a new service, endpoint, background job, or external integration
11- A production incident took too long to diagnose ("we couldn't tell what happened")
12- Setting up or reviewing alerting rules
13- Reviewing a PR that adds I/O, retries, queues, or cross-service calls
14 
15**NOT for:**
16- Diagnosing a failure happening right now — use the `debugging-and-error-recovery` skill (observability is what makes that skill fast next time)
17- Profiling and optimizing measured slowness — use the `performance-optimization` skill
18- Launch-day monitoring checklists and rollback triggers — see the `shipping-and-launch` skill; this skill covers the instrumentation that feeds them
19 
20## Process
21 
22### 1. Define "working" before instrumenting
23 
24Telemetry without a question is noise. Before adding any instrumentation, write down 2–4 questions an on-call engineer will ask about this feature:
25 
26```
27FEATURE: checkout payment retry
28QUESTIONS ON-CALL WILL ASK:
29…