custom-engine

1# Custom Engine e2e fixture
2 
3This directory is a minimal skill-up fixture that exercises the **Custom
4Engine** local and http transports end-to-end. It is consumed by
5`e2e/custom_engine_test.go`.
6 
7`agent.sh` + `evals/eval.yaml` cover the **local** transport: `agent.sh` is a
8deterministic stand-in for a real custom agent that reads the `SessionInput`
9JSON the framework writes to `${input_file}` and emits a fixed `SessionResult`
10on stdout. The case asserts that `final_message` flows from the agent's stdout
11into the report.
12 
13`evals/eval-http.yaml` exercises the **http** transport against the same
14`hello.yaml` case. `TestPipeline_CustomEngine_HTTPTransport` starts an
15in-process `httptest` server, points `${CUSTOM_AGENT_ENDPOINT}` at it, and
16asserts the posted `SessionInput` reaches the server and the returned
17`SessionResult` flows into the report — so the http stand-in agent lives in the
18test rather than in `agent.sh`.
19 
20The `eval-stateful-*.yaml` fixtures exercise opt-in stateful conversations for
21both transports, including session-ID propagation, per-turn post-conditions,
22capture, substitution, and a final response that needs no continuation ID.