What Is an Agent’s Trace?

written by Manu
Agent's trace
The full code for this article is on GitHub. It’s a working Python prototype with the onboarding system and the case records used in this post. You don’t need it to follow along, but it’s there if you want to try it yourself.

In the last article, we built an agent map.

The map captures what we decided before the agent runs. It covers what the agent should do, which regulations apply, what information it uses, and how it accesses that information.

The trace is different.

It shows what happened when the agent ran. What information it received, what it did with that information, and what answer it produced.

So the map is the plan.

The trace is the record.

Let’s see what that looks like with James

James is the high-net-worth client we’ve been following through this series. He sold his software company and plans to bring $5 million into the bank. Of that, $4 million comes from the business sale and $1 million from crypto funds.

The onboarding system uses four specialist agents to check different parts of his case. Their findings go to the Risk Engine, which applies the bank’s risk rules. It assigns James a risk level and flags any gaps that still need attention from the compliance officer.

The Case Summary Agent then takes the Risk Engine’s output and turns it into plain English for the compliance officer. It is the only agent in this system that uses an AI model.

Here’s what it produced for James.

1. OPENING LINE
James Whitmore – High-risk client.

2. WHAT HAS BEEN VERIFIED

- Identity verification is complete.
- No sanctions match was found.
- Wealth documents and bank statements have been provided and are on file.
- The business-sale transaction is supported by the relevant documents and
  the corporate registry.

3. WHAT NEEDS REVIEW

- Political-exposed person (PEP) status: James Whitmore has been identified
  as a PEP, which raises a higher risk of corruption or undue influence.
  Enhanced due-diligence is required to understand the nature of his public
  role and any related connections.

- Crypto-derived funds: He has declared cryptocurrency holdings and supplied
  exchange records, but the source of those crypto funds has not been
  established. Without a clear origin, there is a risk of money-laundering or
  financing illicit activity.

4. WHERE TO FOCUS

- PEP due-diligence: Obtain a detailed description of his current and former
  public positions, any family or close-associate relationships, and the
  purpose of the new account. Request supporting documentation (e.g., official
  appointment letters, public disclosures) and perform a risk-based assessment
  of the political exposure.

- Crypto fund origin: Ask for the full transaction history from the exchange,
  including inbound and outbound transfers, wallet addresses, and any KYC
  information the exchange holds on the counterparties. Verify whether the
  crypto was acquired through legitimate means (e.g., mining, sale of assets,
  investment) and document the source-of-wealth narrative.

These steps will address the outstanding high-risk factors before the case
can move forward.

Let’s see how that summary was generated.

For this, you need agent tracing software such as LangSmith. It records every step the agent takes to produce its final summary.

Open the Case Summary Agent in LangSmith and you can see what it was handed before it wrote anything.

This is the Risk Engine’s output, and it’s the only thing the Case Summary Agent received. Four verified items, two items needing review, and the reasoning behind the HIGH risk signal.

Now put that next to the summary above. Four verified items went in and four came out. Two review items went in and two came out. But the line about corruption in the PEP paragraph isn’t here. The Risk Engine said the status was confirmed and that enhanced due diligence was required, and nothing more. The model added the rest.

Something else is missing too. The Risk Engine’s reasoning names FINTRAC but the summary doesn’t mention it anywhere.

Whether that’s a problem is a question for the next article. What matters right now is that you could spot it at all, and you could only spot it because the trace recorded the input alongside the output.

And it doesn’t stop with one agent.

You can also see the trace for the entire onboarding system.

Here you can see which agents ran, what each one received and returned, and how long each one took. The whole run finished in 1.49 seconds, and the Case Summary Agent accounted for 1.36 of those. Everything else together took less than a tenth of a second.

In a real bank, this trace would be far more detailed. The system would be pulling from more sources, running more agents, and using several AI models across them. Every one of those steps gets recorded, which gives the bank a complete account of what happened during the run.

We now have both sides of the picture.

The map tells us what the agent was supposed to do, but the trace shows us what it actually did when we ran James’s case. With those two pieces in place, we can start evaluating the agent. That’s what the next article is about.

 

Tags: