How to investigate a run

Open a run, trace a result back to the exact node execution that produced it, and read its input, prompt, output, and checks.

When you need this

Use this when a run gave you a result you didn’t expect and you need the real cause, not a guess. The runs screen is built to answer, in order: which step is at fault, what input it got, where that input came from, what prompt was built from it, what the model answered, which checks ran against that answer, and what happens if you change the input or the prompt and rerun just that one step.

Steps

Open a flow’s runs screen and pick a run from the picker at the top — search by run reference, status, or the dataset case it used. If nothing is selected yet, Studio shows the most recent run. Two buttons next to the picker start a new one: Start a run sends you to the flow’s Cases tab to pick a saved case, Enter input manually opens a form on this same screen instead.
The header above the timeline shows the run’s status and mode, when it started, and the spec hash it ran against. A run that finished but lost some steps on the way — a map with on_item_error that went on without one item, say — doesn’t read as a plain green COMPLETED: its status says Completed · 1 step failed in amber, here and in the run picker. If this run was forked from another one, a forked from link takes you there; if the flow’s definition changed since this run happened, a warning says so. Four metric cards give you cost, duration, tokens, and a nodes count with how many failed. If the run is waiting on a person, a panel lists which node, who it’s assigned to, and the deadline — that’s what responding to a review resolves.
Failed steps. Under the metrics, a Failed steps panel lists every execution that failed, even when the run itself completed. Each row names what went wrong in plain words — The model rejected the output type, The model's answer never matched the output type, The model provider returned an error — next to the engine’s error code, then the step with its item, branch, or iteration (assess__look · item 0), the error message, and the engine’s hint on what to change. Open step opens that exact execution in the side panel. If the run failed with an error no step explains — a crash of the engine, a run-level timeout — a red panel above the list shows that error on its own.
Below that is the stage timeline: one card per top-level node, in the order the flow defines them, plus grey “not started” cards for anything that hasn’t run yet. A loop, parallel, map, or switch node’s card holds a matrix instead of a single result — one column per call: one pass of a loop, one branch of a parallel, one item of a map. When some items of a map or branches of a parallel failed, its card carries a badge such as 1 of 5 failed and the failed columns are red; a stage that finished OK with a failure somewhere inside it gets an amber marker, and the stage navigator says OK · 1 failed inside.
When an item of a map fails and the node’s on_item_error policy keeps the run going, the item is not counted as failed: the map’s card reads OK with a yellow 1 replaced (or 1 skipped) mark, and hovering it shows which policy decided. The item’s column says FAILED · replaced in yellow, and its output row shows the value the policy put in its place, captioned Default from <policy> — not a model answer. The run’s status reads Completed · 1 item replaced, and so does the nodes card. See the map node for the policies themselves.
Click any cell in a matrix — input, prompt, output, whichever row you want — and a side panel opens to that exact cell. This is what makes “which execution” a real, clickable thing instead of a guess: the panel’s title is the node’s id, and right under it are the branch, iteration, or item number that identify this one execution among however many times that node ran. That’s the same address the engine itself records for every execution — see the engineering loop for why every run of a step needs one.
If a column belongs to a nested loop, parallel, map, switch, or a called sub-flow, its matrix opens right below as its own panel, labeled Inside <name> — click into that one the same way, and it can open another level under it. That’s how you get from a top-level container all the way down to one specific model call inside a loop inside a parallel branch.
If a node retried after a failure, a list of its failed attempts appears under its matrix: one row per attempt, with what caused it, what the engine did about it, the error message, and a raw excerpt of what the model actually returned. Identical answers inside one response are shown once with a ×N count, and the post-check row folds repeated check results into one line with how many times and over how many attempts they failed. A loop’s card also ends with a line saying why it exited and which pass it kept.
The side panel itself has five tabs: model (which agent, which inference, which actual model answered — and, for a failed call, the error on top: the same plain-words title, message, and hint as in Failed steps, the provider and its HTTP status, and the provider’s raw answer folded under Provider response), input (what this call read, and from which upstream node), prompt (the prompt as it was actually sent — a numbered list of messages — with the original template on disk collapsed underneath it), output (what came back), and checks (every check that ran, pass or fail, the error of a failed call, and for a human step, who it waited on and what they answered). A formatted/raw switch above the panel’s body — same mechanism as the node inspector — flips every section between a readable view and the raw JSON.
Turn a run into a case with To cases. Once you’ve found a wrong answer, lock it in: To cases opens “Draft a case from this run”. The engine turns the run’s input, its context and the outputs of its top-level nodes into a dataset case of this flow, and Hand the case to the chat sends the draft to the chat. The agent keeps only the node_outputs the case needs, names and tags it, and asks you for the expected_output before it writes the file. A flow without a dataset gets a new one with this case first. An experiment that selects the case by its tags then keeps checking the fix.
Compare agents on a step. When the stage you’re on calls a model, the stage navigator shows Compare agents on <step>. It hands the chat a prompt to write a compare or noninferior experiment for that step. The step’s current agents become the baseline, other agents from agents/ become candidate variants, and the cases come from the dataset this run used. See How to write an experiment.
To start a manual run, drag the start and end handles on the node range picker to choose which stages to run, or click a single node to run only that one stage — the same node you just found to be at fault, with a changed prompt or a changed input, is a valid range of one. Studio checks live whether the range you picked has everything it needs. Fill in only the input fields that range actually reads; if you’re starting in the middle of the flow, you also get a JSON field for each earlier node’s output the range depends on, with a shortcut to copy it straight from a previous run. Add run context (date, time_zone, locale, tenant_id) if the flow reads any of it, review the summary, and submit — a manual run always executes live, calling real models and tools.

Example

Open a run of the support_case flow and find drafts, a parallel node: its matrix has one column per model provider drafting a reply side by side. Click the output cell of one column and the side panel opens on that provider’s draft text; switch to the prompt tab in the same panel and you see the exact prompt only that provider received, not the other two.

Now open record, a loop that fills in the case record and checks it against business rules pass by pass. Its matrix has one column per pass — record__extract reads the case, record__validate checks the result. Click record__validate’s output on the first pass and compare it with a later pass: if a rule failed the first time and passed the next, the checks tab shows exactly which rule and why, and the loop’s footer names which pass it kept.

How to investigate a run

When you need this

Steps

Example

See also

Start

Engine

Studio

MCP & CLI

Integrations

Concepts

Reference