Build made the agent, Scale ran it, Govern controlled it. Optimise closes the loop: evaluation (does it still do the right thing?), simulation (does it, when a user pushes back over several turns?), observability (traces, logs and analytics: where do time and tokens go?) and prompt optimisation (use the failures to improve the instructions).

The idea to hold onto: an opinion about your agent is not a measurement. Every change you make today follows the same four moves: measure, change one thing, measure again, keep it only if the number agrees.

There are three ways to score an agent, and the first is the trap:

1. String matching (response_match_score, ROUGE). Compares words with a reference. Cheap and deterministic, but a correct answer phrased differently scores low. It is the default if you configure nothing, which is why a first eval run so often fails for the wrong reason.
2. Trajectory matching (tool_trajectory_avg_score). Compares the tools called and their arguments with an expected list. Here it can never pass: the orchestrator calls AgentTools whose arguments are model-written free text, so the expected arguments can never equal the real ones.
3. LLM-as-judge with rubrics (rubric_based_*, final_response_match_v2). A second model reads the conversation and the tool calls and scores each rubric you wrote in plain English, for example "never calls create_post before the user approves the exact draft". This is what evals/eval_config.json uses.

Rubrics are tests written as sentences. The judge is a model too, so a rubric has to be specific enough that two readers would score a reply the same way. num_samples asks the judge several times and aggregates, which smooths its noise at the cost of more calls.

A scenario is a starting_prompt (the fixed first message) and a conversation_plan (what the user wants, how they react if the agent refuses, when they give up). During the eval an LLM plays the user turn by turn from that plan. Scoring then looks at the whole conversation, so it can catch a failure that only appears on turn three: the agent that holds firm on turn one and gives in on turn three.

Two parts of Google's tooling are involved, and only one needs the cloud:

• Generating scenarios. client.evals.generate_conversation_scenarios(...) in the Gen AI evaluation SDK writes scenarios for an agent you describe. It works in a fresh personal project with no allowlist.
• Playing them. ADK's own adk eval runs a scenario case with a user_simulator_config, locally. No cloud service is needed beyond Gemini.

Generate once, then read them and freeze the file. A generated set that changes every run is not a regression test. You review it the way you would review a pull request, and commit it.

You pay for input tokens (everything sent on every call: instructions, history, tool results), output tokens (including a thinking model's reasoning, billed as output), and tool or runtime time. In a multi-agent chain the same context is paid for again at every hop, so the cheapest tokens are the ones you stop sending.

Three levers, cheapest to try first: send less (shorter instructions, state instead of repeated text, trimming history), think less (a lower thinking budget on agents that only route), use a smaller model where quality shows least. Per-agent models are already environment variables, so this step changes no code.

An LLM agent and an LLM judge are both non-deterministic. Before you trust a difference between two configurations, you need to know how big the difference is between two runs of the same one. Practical rules: run each candidate several times; treat a case that flips as unstable, not as passing or failing; raise the judge's num_samples for the cases you gate on; and prefer a deterministic metric wherever the behaviour allows one (as Lab 3 did for the injection refusal). A gate built on a flaky case will either block good deploys or, worse, teach the team to re-run until it is green.

Three views of the same turn, each good at something different:

• A trace (Cloud Trace, the console's Traces tab) is the timeline: one span per agent run, model call and tool call, nested. Best for "why was this one request slow?".
• Logs are the text of what happened. Best for "what did the model actually see?". They only hold prompts and replies if you switch that on, because they can hold personal data.
• An analytics table (BigQuery Agent Analytics) is one row per event, queryable with SQL across every turn. Best for "which agent costs the most, and did that change after Tuesday's deploy?".

This step wires the third, because it is the one you can verify from your own laptop.

Anything in an agent's instruction is paid for on every single model call, forever. Anything in session state (written with output_key, read back with {research_notes}) travels only to the agents that ask for it. Here research_agent writes its summary to state, draft_agent reads it, and the orchestrator never carries it. The orchestrator's long rule list, on the other hand, rides on every call it makes.

Two further levers exist but are not wired here: memory retrieval (fetch the one relevant memory instead of keeping history in context) is already how load_memory works on the deployed agent, and ADK's App accepts an events-compaction setting that summarises old turns. Neither is a free win: a summary can drop the detail a later turn needed, which is exactly what an eval would catch.

The pipeline has two steps in one file, cloudbuild.yaml. eval-gate installs dependencies, starts the mock memory agent, runs the golden set, then reads the "Tests failed: N" line itself and exits non-zero if N is above zero or the line is missing. deploy only runs if the first step succeeded. Cloud Build stops at the first failing step, so a red eval means no new revision.

Remember Step 2: the gate works only because it does not trust adk eval's exit code. It also fails closed. A gate that cannot parse a result must block, not wave through.