ConceptA tool I would build. Not client work.
A run console for AI agents.
Your agent takes actions on real customer accounts. This is the screen where the person who answers for them can see what it did, step by step, and stop it before it does the wrong thing.
Concept
Not client work- Drawn for
- A funded startup shipping an agent, and the operations lead who signs off on what it does.
- The problem
- The agent refunded someone overnight and nobody can say why. The only record is a trace built for the engineer who wrote it.
- The money
- 32% of 1,340 people building agents name quality as a top barrier to putting one in production, and 59.8% rely on a person reviewing the output. LangChain, State of Agent Engineering (opens in a new tab).
- Belongs to
- Product and MVP
Every company, person and number on these screens is made up. Nobody has bought this yet.
Live demo
Drag the run backwards.
DemoDrag the playhead back along the track. The agent’s notes and the customer record go back with it.
Refund request 4821
Needs approval- Trigger
- Step
- 7 of 7
- Cost
- €0.040
- Time
- 0:41
Asks a personPerson · approval.request
Steps
- In
- The proposal, the evidence and the draft.
- Out
- Sent to the approvals queue.
- Took
- 3.4 s, no model cost
Agent notesWritten by AI
- She says the tent arrived with a snapped pole and wants her money back.
- Plan: find the order, read the refund policy, look at the photo, then decide.
- Order A-20931. Ridgeline 2 tent, €129.00, delivered 3 November.
- Policy 4.2: damaged on arrival, reported within 30 days with a photo, refund in full.
- The photo shows a pole section broken at the joint. It matches what she wrote.
- Delivered 8 days ago, so she is inside the 30 days. She qualifies for a full refund.
- Reply drafted. No replacement offered: the tent is out of stock until January.
- €129.00 is over my €100 limit. I can’t send this myself, so I’m asking a person.
Customer record
- Open request
- Refund, by email, 14:03step 1
- Reason
- Damaged on arrivalstep 1
- Order A-20931
- Delivered 3 Novbefore this run
- Evidence
- 1 photo, checkedstep 4
- Refund
- €129.00, waiting for a personstep 6
- Reply
- Draft ready, not sentstep 5
Waiting for you. A €129.00 refund to Mara Lindqvist is over this agent’s €100 limit.
Concept
Run timeline
One run from the first email to the refund. The run is a recording, so any moment of it can be shown again.
The rest of the tool
Four more screens.
- FernhillRunsApprovals3EvalsHealthAgentsRefund agentOrder agentWarranty agentSam OkaforOperations lead
Refund €129.00 for Mara Lindqvist
Refund agent · run 4821 · waiting 2 min
Why it stopped. Refunds above €100 need a person.
What changes if you approve
Field Now After Order A-20931 DeliveredRefunded Refund None€129.00 to the original payment method Stock Unchanged Unchanged. She keeps the tent What the agent checked
- Order A-20931 found, delivered 3 November
- Policy 4.2 applies: reported within 30 days, with a photo
- The photo matches what she wrote
Drafted by AI
Edit it here before you approve.
What Mara wrote
“The tent came last week and one of the poles is snapped at the joint. Photo attached. I haven’t used it. Can I have my money back? I need a tent this weekend, so I can’t wait for a new pole.”
Email · 1 photo The run so far7 steps, 41 s, €0.04
- Plan
- Order
- Policy
- Photo
- Decide and draft
- Refund limit
- Ask you
The agent does nothing until you choose.
Concept
Approval queue
What the agent wants to change, as a before and after. It does nothing until a person chooses.
1 / 4 - FernhillRunsApprovals3EvalsHealthAgentsRefund agentOrder agentWarranty agentSam OkaforOperations leadAll today128Needs approval3Failed2Running1
Run Agent Trigger Status Why Steps Cost Time Started Refund request4821 Refund agent Email Needs approval Over the €100 limit 7 €0.04 41 s 14:03 Address change4820 Order agent Chat Done 4 €0.01 12 s 14:01 Refund request4819 Refund agent Email Running Step 3, checking the policy 3 €0.02 18 s 14:01 Where is my order4818 Order agent Chat Done 3 €0.01 8 s 13:58 Warranty claim4817 Warranty agent Email Failed Carrier did not answer 5 €0.03 27 s 13:55 Refund request4816 Refund agent Email Done 6 €0.03 33 s 13:51 Cancel order4815 Order agent Chat Needs approval Order already shipped 5 €0.02 22 s 13:49 Size exchange4814 Order agent Email Done 8 €0.04 46 s 13:44 Where is my order4813 Order agent Chat Done 3 €0.01 7 s 13:40 Refund request4812 Refund agent Email Done 6 €0.03 30 s 13:37 Invoice copy4811 Order agent Email Done 2 €0.01 5 s 13:31 Warranty claim4810 Warranty agent Email Needs approval 12 days out of warranty 9 €0.05 58 s 13:26 Address change4809 Order agent Chat Failed Order not found 2 €0.01 4 s 13:20 Refund request4808 Refund agent Email Done 6 €0.03 35 s 13:12 Where is my order4807 Order agent Chat Done 3 €0.01 9 s 13:09 Size exchange4806 Order agent Email Done 7 €0.04 44 s 13:02 Invoice copy4805 Order agent Chat Done 2 €0.01 4 s 12:58 128 runs today€4.12 spentMedian 24 sAll three agents liveConcept
Runs
Every run today with its status, steps, cost and time. The ones that need a person say why.
2 / 4 - FernhillRunsApprovals3EvalsHealthAgentsRefund agentOrder agentWarranty agentSam OkaforOperations lead
Case What it checks v12Previous v13Live v14Candidate Passed, of 24 19 21 23 Second refund for the same order Must refuse: already refunded Fail Fail Fail Photo shows a different product Must ask for a new photo Fail Fail Pass Customer writes in Dutch Must answer in Dutch Pass Fail Pass Order older than 30 days Must refuse and explain Fail Pass Pass One item of three is damaged Must refund that item only Fail Pass Pass Angry tone, no order number Must ask for the number Fail Pass Pass Damaged on arrival, with photo Must refund in full Pass Pass Pass Refund above the €100 limit Must stop and ask a person Pass Pass Pass No photo attached Must ask for one Pass Pass Pass Item out of stock Must not offer a replacement Pass Pass Pass Asks for store credit instead Must offer credit Pass Pass Pass Email is not about a refund Must hand over, no action Pass Pass Pass Pinned casefails in v14Second refund for the same order
- Given
- An order that was refunded on 2 November. The customer asks again on 9 November.
- Expected
- Refuse. Say the refund was already sent, and when.
- Got
- Proposed a second refund of €129.00.
- Caught by
- The €100 limit, so a person saw it. Nothing was paid twice.
Showing 12 of 24 cases, failures firstLast full run 13:40, 2 min 12 sConcept
Evals
The same 24 test cases against every version. A failing case is pinned until it passes.
3 / 4 - FernhillRunsApprovals3EvalsHealthAgentsRefund agentOrder agentWarranty agentSam OkaforOperations leadRuns today128131 yesterdayFailed today1.6%2 of 128Cost per run€0.032€4.12 todayWait for a person3 min 10 smedian, 3 waiting nowRuns per day29 October to 11 NovemberRuns Failed, shown 3× tallWhy runs failedlast 7 days, 15 runs
Carrier did not answer in time6 of them on 5 November
8Order not foundWrong email on the request
4Model declined to answerAbusive message
2Stuck in a loop, stopped by a guardrailWarranty agent
1
By agenttodayAgent Runs Failed Cost Per run Waiting on a person Refund agent 61 0 €2.31 €0.038 1 waiting Order agent 58 1 €1.42 €0.024 1 waiting Warranty agent 9 1 €0.39 €0.043 1 waiting Concept
Health
Runs a day, the share that failed and why, cost per run, and how long people keep the agent waiting.
4 / 4
The money
What it would be worth.
Fewer wrong actions reach a customer, and when one does, you find the step that caused it in a minute.
I found no public figure for hours saved, so I won’t invent one. Count how many actions wait for a person each week and how long each takes to understand. That number is the business case, and it is yours.
What you can buy today
- LangSmith (opens in a new tab)$39 per seat a month
- Langfuse (opens in a new tab)$29 to $2,499 a month
- Braintrust (opens in a new tab)$249 a month
List prices from each vendor’s own page, checked on 11 November 2026. They are good, and they are built for engineers. A custom console is the better buy when the person approving is not one, and needs the run in the words of the business.
What approving costs you now
Type your numbers and the sum appears here.
This is the cost of the reading, on your numbers. How much of it a clearer console removes is something to measure in week one, not something I can promise here.
Want this for real?
Tell me what yours has to do. In 20 minutes you will know whether it is worth building, and what I would cut from this drawing.
I would scope it as a product or MVP: from €24k, usually 4–8 weeks. What that includes