Author: Manu

In the last article, we built an agent map.

The map captures what we decided before the agent runs. It covers what the agent should do, which regulations apply, what information it uses, and how it accesses that information.

The trace is different.

It shows what happened when the agent ran. What information it received, what it did with that information, and what answer it produced.

So the map is the plan.

The trace is the record.

Let’s see what that looks like with James

James is the high-net-worth client we’ve been following through this series. He sold his software company and plans to bring $5 million into the bank. Of that, $4 million comes from the business sale and $1 million from crypto funds.

The onboarding system uses four specialist agents to check different parts of his case. Their findings go to the Risk Engine, which applies the bank’s risk rules. It assigns James a risk level and flags any gaps that still need attention from the compliance officer.

The Case Summary Agent then takes the Risk Engine’s output and turns it into plain English for the compliance officer. It is the only agent in this system that uses an AI model.

Here’s what it produced for James.

1. OPENING LINE
James Whitmore – High-risk client.

2. WHAT HAS BEEN VERIFIED

- Identity verification is complete.
- No sanctions match was found.
- Wealth documents and bank statements have been provided and are on file.
- The business-sale transaction is supported by the relevant documents and
  the corporate registry.

3. WHAT NEEDS REVIEW

- Political-exposed person (PEP) status: James Whitmore has been identified
  as a PEP, which raises a higher risk of corruption or undue influence.
  Enhanced due-diligence is required to understand the nature of his public
  role and any related connections.

- Crypto-derived funds: He has declared cryptocurrency holdings and supplied
  exchange records, but the source of those crypto funds has not been
  established. Without a clear origin, there is a risk of money-laundering or
  financing illicit activity.

4. WHERE TO FOCUS

- PEP due-diligence: Obtain a detailed description of his current and former
  public positions, any family or close-associate relationships, and the
  purpose of the new account. Request supporting documentation (e.g., official
  appointment letters, public disclosures) and perform a risk-based assessment
  of the political exposure.

- Crypto fund origin: Ask for the full transaction history from the exchange,
  including inbound and outbound transfers, wallet addresses, and any KYC
  information the exchange holds on the counterparties. Verify whether the
  crypto was acquired through legitimate means (e.g., mining, sale of assets,
  investment) and document the source-of-wealth narrative.

These steps will address the outstanding high-risk factors before the case
can move forward.

Let’s see how that summary was generated.

For this, you need agent tracing software such as LangSmith. It records every step the agent takes to produce its final summary.

Open the Case Summary Agent in LangSmith and you can see what it was handed before it wrote anything.

This is the Risk Engine’s output, and it’s the only thing the Case Summary Agent received. Four verified items, two items needing review, and the reasoning behind the HIGH risk signal.

Now put that next to the summary above. Four verified items went in and four came out. Two review items went in and two came out. But the line about corruption in the PEP paragraph isn’t here. The Risk Engine said the status was confirmed and that enhanced due diligence was required, and nothing more. The model added the rest.

Something else is missing too. The Risk Engine’s reasoning names FINTRAC but the summary doesn’t mention it anywhere.

Whether that’s a problem is a question for the next article. What matters right now is that you could spot it at all, and you could only spot it because the trace recorded the input alongside the output.

And it doesn’t stop with one agent.

You can also see the trace for the entire onboarding system.

Here you can see which agents ran, what each one received and returned, and how long each one took. The whole run finished in 1.49 seconds, and the Case Summary Agent accounted for 1.36 of those. Everything else together took less than a tenth of a second.

In a real bank, this trace would be far more detailed. The system would be pulling from more sources, running more agents, and using several AI models across them. Every one of those steps gets recorded, which gives the bank a complete account of what happened during the run.

We now have both sides of the picture.

The map tells us what the agent was supposed to do, but the trace shows us what it actually did when we ran James’s case. With those two pieces in place, we can start evaluating the agent. That’s what the next article is about.

To evaluate an AI agent, its final answer isn’t enough.

You also need to know what information it used, whether it followed the instructions it was given, and where that information came from.

To see all of that, you first need a map.

So what is that map?

An agent map holds four things.

• What the agent is expected to do
• What regulations it has to follow
• What information it uses
• How it accesses that information

Let’s build a map for the onboarding system we’ve been using throughout this series. 

The onboarding system is what we built to bring a high net worth client onto a bank’s books.

The orchestrator routes the client’s information to four specialist agents. Their findings go to the risk engine, which applies the bank’s rules and classifies the client as HIGH, MEDIUM, or LOW risk. The case summary agent then translates those findings into plain English for the compliance officer. 

Only the case summary agent uses an AI model, so that’s the one we’ll build a map for.

Let’s start with the first part of the map.

What the agent is expected to do

At a high level, the case summary agent translates the risk engine’s output into plain English. Let’s break that requirement down into its individual parts.

• Structure. The summary splits into four parts. An opening line, what has been verified, what needs review, and where to focus.
• Coverage. Every verified item appears, and every item needing review appears. Nothing gets skipped.
• Explanation. Each item needing review comes with a reason why it matters, written in plain English.
• Next steps. Every unresolved item tells the compliance officer what to do next. 
• No internal codes. The officer reads English, not PEP_CONFIRMED or WEALTH_SUPPORTED.
• No recommendation. Approve and reject are words the agent never uses.
• Length. Under 250 words.
• Only what it is given. Every statement comes from something the risk engine supplied. The agent adds nothing of its own and softens nothing it received. If the risk engine says a case cannot proceed, the summary can’t turn that into a suggestion.

A case summary agent sounds simple until you write down everything it’s expected to do.

All of those requirements become part of the agent’s prompt. 

This prompt can still be improved. Examples of good summaries would make it stronger, and there are prompt engineering techniques that would sharpen it further. We’ll come back to both in a later article. For now, these requirements define what the agent has been asked to deliver.

What regulations it has to follow

An agent doesn’t just follow the instructions in its prompt. It also has to follow the regulations that apply to it, and those regulations shape how the agent is built.

In Canada, banks using AI agents fall under OSFI Guideline E-23. Among other things, it requires AI-assisted output to be explainable and documented. That means every input the case summary agent receives and every summary it produces has to be recorded, so someone who didn’t build the agent can still review its work. Most other countries already have similar requirements in place, and those that don’t are working towards them.

Two more regulatory requirements apply to this agent.

The compliance officer makes the decision, not the agent. The bank remains accountable for every onboarding decision, so nothing the agent writes can pre-empt the compliance officer’s judgement. That is where the rule against using words like approve or reject comes from. It was never a style choice.

Another requirement comes from the PCMLTFA. If a client is flagged in a sanctions match, the summary cannot instruct the compliance officer to ask the client about it, because doing so could amount to tipping off the client, which is an offence under the Act.

All of that applies to an agent whose only job is writing a case summary.

If the identity verification agent also used AI, it would inherit a completely different set of regulations. The Canadian PCMLTFA specifies how a client’s identity can be verified, which methods are acceptable, and when enhanced verification is required. The agent would also have to record which verification method it used for each client and retain that information for years after the relationship ends.

Even within the same onboarding system, each AI agent has its own regulatory requirements. 

That’s why every AI agent gets its own map.

What information it uses  

Every agent gets its information from somewhere and uses that information to do its job.

Our case summary agent receives a single input from the risk engine and turns it into a plain English summary.

In a real bank, a single agent might receive information from several places at the same time, such as:

• Findings from other agents
• Internal bank systems
• External vendors
• Public registries
• Documents uploaded by the client

All of those information sources need to be captured in the agent’s map.

How it accesses that information

Knowing where the information sits doesn’t mean the agent can reach it, and what gives it that reach is a tool.

A tool is simply a piece of code the agent can call to retrieve information or perform an action.

If the screening agent used AI, one of its tools might check the sanctions provider’s database and return the result to the agent.

Another tool might update the case management system after a review is complete. In that case, the tool performs an action instead of retrieving information.

Our case summary agent doesn’t use any tools because it simply receives the risk engine’s output and turns it into plain English.

One more thing belongs in this part of the map, and that’s the model itself. So does the version of the prompt it runs on.

The model decides which tool to call and when. It’s also what turns the risk engine’s findings into the sentences the compliance officer reads. Our case summary agent runs on gpt-oss-120b, so the model and its version number become part of the map. 

Swap that model out and you’ve changed the agent, even though nothing else in the map has moved. Change a single line in the prompt and the same thing happens. That’s why banks record both versions, and evaluate the agent again every time either one changes.

With all four parts written down, the map looks like this.

Four rows is all it takes, and that’s because the case summary agent is a simple agent. The same onboarding system inside a bank might have several AI agents, and every one of them needs its own map.

That sounds like a lot of work, but it isn’t.

Most of this information already existed when you built the first version of the system. During the mapping process, all you’re doing is bringing it together in one place, agent by agent. Once you’ve done that for every AI agent in the system, you’re ready to evaluate them.

Now, when the agent produces an answer, you have something to compare it against.

Instead of asking whether the answer sounds right, you can check whether the agent behaved the way you designed it to.

What the map can’t tell you is whether the agent actually did those things.

When an AI agent produces the wrong answer, you need to see every step that led to it. Which information it actually used, whether it followed the instructions it was given, and where each piece of that information came from.

That record is called a trace, and that’s what we’ll look at in the next article.

You already know how to test software. You feed it an input, check the output against the answer you expected, and know right away whether it passed or failed.

AI agents are different. Run the same input twice and you can get two different answers, both of them perfectly reasonable.

So what do you compare the output against?

The easiest way to answer that question is to go back to James’s onboarding.

Here’s the system we’ve been building throughout this series.

The specialist agents have already done their work and passed their findings to the bank’s risk engine.

The risk engine applies the bank’s risk rules and produces its output as JSON, a collection of codes and fields that isn’t easy for a compliance officer to read.

A case summary agent translates that JSON into plain English for the compliance officer.

Only one agent in this system actually uses an AI model: the case summary agent. 

The specialist agents and the risk engine all run on rules the bank has written down. Give them the same input and they return the same output every time. That makes them straightforward to test. 

The case summary agent is the only part of this system that uses an AI model. In this prototype, that’s GPT-OSS 120B through Groq.

Here is the prompt the case summary agent uses.

The prompt is detailed. It tells the model what to write, how to structure it, and what to leave out.

Now let me run the onboarding system twice on James’s case.

Here are both outputs.

The first summary tells her to obtain documentation from James regarding the source of the crypto funds. The second tells her to investigate the crypto exchange’s records.

The regulatory references change too. The first summary refers to FINTRAC guidelines twice, and the second never mentions them at all, even though the Risk Engine explicitly included them.

Nothing changed between those two runs. The same prompt produced two different summaries. 

So which one is correct?

We can’t answer that question yet. Nobody has ever written down what a correct summary looks like.

And if we can’t define “correct,” neither can the AI.

So before we can evaluate the model, we first have to define what a good answer looks like. 

For James, the risk engine produced this output. It typically arrives as messy JSON, so I’ve cleaned it up here to make it easier to read. 

From that output, the bank can define the evaluation criteria. 

A correct summary must:

• State the risk signal and explain why it was assigned
• Include every item the risk engine flagged for review
• Preserve any regulatory references the risk engine supplied
• Tell the compliance officer what to do next in clear, actionable language
• Avoid exposing internal system codes

Those criteria don’t come from the team building the agent. They come from the compliance officers who already write these summaries today.

Now go back to the two outputs.

The second summary removes the FINTRAC reference even though the Risk Engine supplied it, so it fails the evaluation while the first summary passes.

That’s the real value of an AI eval. 

You aren’t simply saying one answer reads better than another.

You’re measuring every answer against the same agreed standard, which tells you exactly what broke and what needs fixing.

But James is just one client.

A real bank processes hundreds of different clients. Some declare crypto assets, others are politically exposed persons, and many have several sources of wealth that all need verifying.

So the bank builds a large collection of client cases covering that range and runs the agent against every one of them. 

Some summaries meet the criteria and others don’t. Each summary that fails the evaluation tells the team exactly what needs fixing.

The problem is that no compliance team can manually review thousands of summaries every time the agent changes. So the bank automates the entire evaluation process. 

Every time the prompt or the model changes, the agent runs against the same collection of client cases. The system checks whether each summary meets the criteria and reports anything that breaks before the agent goes live.

So why does all of this matter so much in banking?

Go back to the evaluation criteria for a moment.

One line says the summary must include every item the Risk Engine flagged for review.

Now imagine that line was never there.

The Risk Engine correctly flags that the origin of James’s crypto funds still needs to be reviewed, but the case summary agent leaves that point out. 

The compliance officer never sees it. She investigates the PEP finding, resolves it, and approves James’s application. As far as she’s concerned, every issue has been addressed. 

Years later, regulators review the case and ask why the bank never looked into the source of James’s crypto funds.

The bank can’t answer that the AI forgot to mention it. From the regulator’s perspective, the decision wasn’t made by the AI. It was made by the bank, and the AI was simply another system the bank chose to use.

The bank may now face millions of dollars in regulatory penalties because it approved James without investigating the source of his crypto funds.

That’s why AI evaluations matter so much in banking. A missing sentence in a summary isn’t just a model error you fix in the next sprint. It’s a compliance failure with real financial consequences.

Those consequences are exactly why AI agents have to be evaluated differently.

Traditional software is easy to test because you already know the correct answer. AI doesn’t give you that certainty. 

Instead, the bank defines what a good answer must contain and checks every summary against those criteria.

That is an AI evaluation.

In banking, it isn’t just another testing exercise. It’s how you know your AI agent is ready to work with real customers

Imagine your bank hires an experienced analyst to help onboard new clients.

This person knows banking inside out. They have read every article, every book, and every regulation on banking.

And the reading doesn’t stop at banking.

They have also patiently studied an enormous amount of publicly available information.

On paper, this is the perfect hire.

Then the first day comes.

The first client application lands on their desk. It’s from James.  

They know how banks onboard clients. But they just don’t know where to start with this application.

So what went wrong?

Buying an AI model works the same way.

An AI model has learned from an enormous amount of publicly available information. Ask it almost anything about banking, and it will answer. But it still cannot do useful work for your bank, just like your brilliant analyst couldn’t.

Because it has never seen a single customer file, read your internal documents, or learned how your bank actually works.

So what’s the first thing you do?

You point the analyst to a shelf. It contains everything they need to understand how your bank works—client onboarding procedures, internal policies, and the regulations the bank follows. 

Now the analyst knows exactly where to look when they need one of the bank’s procedures. 

But you don’t ask them to read the whole shelf and hold it in their head.

You let them pull down one manual at the moment they need it.

If James mentions in his application that he recently sold his software company and wants to move several million dollars into a new account, that immediately tells the analyst where to look. They pull down the bank’s source-of-funds policy, read that procedure, and continue with the case. 

That simple act is exactly how Retrieval-Augmented Generation works.

You don’t expect the model to carry every document in its memory.

Instead, you build a system around it that retrieves the right document when it’s needed. If James mentions a business sale in his application, the system retrieves the bank’s source-of-funds policy, and the model uses that policy to respond.

That is one of the building blocks of an AI agent, and it’s called RAG.

But the shelf only holds the bank’s own rules.

It says nothing about James himself.

Now the analyst needs information that isn’t written in any policy. They need James’s identity documents to confirm he is who he claims to be. They need to check his name against sanctions lists and politically exposed person registries.

Some of what the analyst needs lives inside the bank. Some of it comes from outside. Either way, they can’t reach any of it on their own.

So the bank gives the analyst tools to access those systems. One tool opens the customer records. Another searches the sanctions database. A third contacts an external identity verification provider. 

With those tools, the analyst can pull up James’s file, check whether his name appears on a sanctions list, and confirm he is who he says he is. 

The AI model works the same way.

On its own, it can only reason from what it already knows. A tool grants it access to one of your bank’s systems, allowing it to retrieve James’s real file or check the latest sanctions list, rather than relying on memory alone. 

But looking things up is only half the job. 

The analyst also has to act on what they find. They might open an onboarding case, record their findings, request another document from James, or flag his application for a human to review. 

A model, on its own, can’t do any of that.

The moment you give it tools that let it retrieve information and take action, it becomes an AI agent.

That is the second building block of an AI agent: tools.

Your analyst can finally do real work. 

But hand them James’s application again tomorrow, and they have forgotten everything about it. 

They still understand banking. What they no longer remember is James, the client they worked on just yesterday. 

An AI model forgets in the same way.

It keeps all the general knowledge it came with. What it doesn’t keep is anything it picked up while working on James’s case.

So why does it seem to remember?

Because each time the model is asked to continue with James’s application, it is handed everything that happened earlier in the case. The model isn’t recalling any of it. It is simply reading through it again before producing its next answer.

That is what we call short-term memory.

But onboarding James rarely happens in one sitting.

One day the bank verifies James’s identity. The next day, it reviews his source of funds. These two checks happen on separate days. Without memory, the model starts fresh each time and forgets everything from the day before.

So you give the model its own filing cabinet, just as you would for the analyst.

When the day’s work is done, the model writes the important details into that cabinet. The next time James’s application comes in, it opens the same folder and picks up right where it left off.

That stored record is long-term memory.

This is the third building block of an AI agent. It’s called memory.

Your analyst now has everything they need. They know your bank’s procedures, they can reach its systems, act inside them, and carry a case forward from one day to the next. So you hand them James’s application.

But James isn’t a simple client. His identity has to be verified. His name has to be screened against sanctions lists. The sale of his software company has to be validated. You could ask one analyst to perform all of those checks, one after another.

Or you could build a team. 

One analyst verifies identity. Another handles sanctions screening. A third reviews the source of funds. A fourth validates the business sale. 

The best part is that none of those checks depend on the others. Instead of waiting in line, they all run at once, reducing work that once took days to minutes.

A team, though, needs a manager.

Someone has to read James’s application, decide which analysts are needed, give each one the right information, and bring all the findings back together.

In an AI agent, that manager is called the orchestration layer.

It decides which specialists run and gives each one exactly what it needs. Their findings then go somewhere else to be brought together, which is a separate job with its own rules.

That is the fourth building block of an AI agent. It’s called orchestration.

Now the picture is complete.

On its own, the model is the analyst on their first day, brilliant and unable to help you.

Give it your bank’s documents through RAG, and it finally knows how your bank works. Add tools, and it can reach your systems and act inside them. Add memory, and it carries a case from one day to the next.

Put an orchestrator above a team of these analysts, and the work that once crawled through one desk now moves as a coordinated team.

Put those building blocks together, and the brilliant analyst on day one becomes someone who can actually onboard James.

That’s an AI agent.

Building one is the easy part. Proving it works well enough to put in front of a compliance officer is a different problem, and that’s what the rest of this series is about.

Let’s say you’ve just finished building a banking AI agent that onboards high-net-worth clients. 

You’ve spent weeks rigorously testing it, and the pilot confirms that the agent handles complex cases correctly and cuts onboarding time significantly.  

Leadership is eager to roll it out. 

You send the agent to the review team for final approval before it goes live.

A few days later, the response comes back saying that the agent cannot go live. 

You’re shocked because the agent works exactly as expected.

So why wasn’t it approved?

To answer that, you first need to understand who this review team is and what they actually do.

Inside every bank is a team responsible for reviewing AI agents before they are allowed to go live. They sit between the developers who build the agents and the regulators who audit the bank.

In many banks, this team is called the Model Risk team. 

Their job is to make sure every AI decision can be explained if a regulator asks about it.

Imagine a regulator walks into the bank six months later and asks, “Show me exactly how this high-net-worth client was onboarded.”

The bank has to answer that question clearly. That means answering questions such as:

• Why was this decision made? 

• Which version of the AI model made that decision? 

• What evidence did the model use? 

You’re probably wondering what could be missing. After all, observability tools like LangSmith already capture every step the agent took.

But the model risk team is asking something else entirely.

Let me show you what they are looking for, using the onboarding agentic AI system I built as a working prototype for this series. If you want to understand how the complete system works, the full walkthrough is here.

Take a client like James, who sold his software company for $18 million and wants to transfer $5 million into a new Canadian bank account.

The onboarding AI system processes James’s application using several specialist agents.

One verifies his identity.

Another screens him against sanctions and PEP registries.

Others review his wealth, source of funds, and business history.

Together, those agents assess James’s application and prepare a complete case for the compliance officer.

The case shows how risky James is as a client, what has already been verified, and what still needs the officer’s attention.

When the system finishes processing James’s application, it leaves behind a complete record of every step it took

Here is what the agent trace looks like in LangSmith, the tool we use to monitor the onboarding system.

You can see which agents ran, how long each one took, and what each agent received and returned. The four specialist agents finish in a few milliseconds. Almost all the run time is the single AI model call at the end.

Now, let’s look at the case summary agent’s output for James, copied exactly from the trace.

1. OPENING LINE
James Whitmore – High-risk client.

2. WHAT HAS BEEN VERIFIED

- Identity verification is complete.
- No sanctions match was found.
- Wealth documents and bank statements have been provided and are on file.
- The business-sale transaction is supported by the relevant documents and
  the corporate registry.

3. WHAT NEEDS REVIEW

- Political-exposed person (PEP) status: James Whitmore has been identified
  as a PEP, which raises a higher risk of corruption or undue influence.
  Enhanced due-diligence is required to understand the nature of his public
  role and any related connections.

- Crypto-derived funds: He has declared cryptocurrency holdings and supplied
  exchange records, but the source of those crypto funds has not been
  established. Without a clear origin, there is a risk of money-laundering or
  financing illicit activity.

4. WHERE TO FOCUS

- PEP due-diligence: Obtain a detailed description of his current and former
  public positions, any family or close-associate relationships, and the
  purpose of the new account. Request supporting documentation (e.g., official
  appointment letters, public disclosures) and perform a risk-based assessment
  of the political exposure.

- Crypto fund origin: Ask for the full transaction history from the exchange,
  including inbound and outbound transfers, wallet addresses, and any KYC
  information the exchange holds on the counterparties. Verify whether the
  crypto was acquired through legitimate means (e.g., mining, sale of assets,
  investment) and document the source-of-wealth narrative.

These steps will address the outstanding high-risk factors before the case
can move forward.

The summary tells the bank what has already been verified, what still needs attention, and where the compliance officer should focus. For James, those areas are his PEP status and the source of his crypto funds.

At first glance, it looks like the agent trace captured everything.

But the model risk team is not looking at what the agents decided. What they want to know is what produced each decision, and whether the bank can recreate it months or even years later.

That means answering questions like:

• Which version of the AI model generated the summary? 
• Which prompt version was active? 
• Which documents and external data sources were used? 
• Which API responses contributed to the decision?

LangSmith does not capture all of that because much of it lives in external systems.

Now imagine if each of the agents above also used an AI model instead of following fixed rules.

The identity agent might use one model, the screening agent another, and the wealth agent a third.

Each would run on a different model version, follow its own prompt, and call a different set of external services.

Every one of those inputs needs to be recorded with its version.

The reason for doing that is simple.

If a regulator asks six months later how James was onboarded, the bank must be able to reproduce that decision using the exact models, prompts, documents, and data sources that were active at the time. 

That is what the model risk team needs before they approve the system to go live.

Now you know why the review team rejected the agent.

It wasn’t because the AI made the wrong decision. The tests passed. The pilot was successful. The agent worked exactly as expected.

The problem was that the bank couldn’t fully explain how those decisions were produced. The agent recorded what it decided, but it didn’t capture everything the model risk team needed to recreate those decisions months or even years later.

That’s the difference between building an AI agent and deploying one inside a bank.

An engineering trace helps developers understand how the system ran. A regulatory record helps the bank explain every decision to a regulator using the exact models, prompts, documents, and data sources that were active at the time.

Until both records exist, the Model Risk team has only one answer.

The agent works. But it still can’t go live.

Most articles explain agentic AI as a technology topic, but very few explain what happens when you apply that technology inside a bank.

Banking is a heavily regulated industry because every decision can have serious consequences for both the customer and the bank. That’s why every decision an AI agent makes needs a complete paper trail.

Months or even years later, a regulator can walk into the bank and ask, “Explain exactly how this decision was made.”

The bank has to be able to answer that question.

Let’s see how that works using one real example.

Meet James

James sold his software company last year for $18 million. He now wants to open a bank account in Canada and transfer $5 million into it. Four million comes from the sale of his business, while the remaining $1 million comes from crypto holdings.

From James’s point of view, everything seems straightforward. He has the money, the documents to back it up, and he’s bringing millions of dollars to the bank.

Yet the bank still can’t open the account.

Why?

Because before the bank can accept James as a client, it has to answer a series of questions.

Did James really sell his software company?

Can the bank trace the $4 million back to that sale?

Can James show where the remaining $1 million in crypto came from?

Has he been screened against sanctions and politically exposed person lists?

A single AI model can answer none of those questions.

Instead, the bank breaks the work into smaller tasks. Each task is handled by a specialist AI agent responsible for one part of the onboarding process.

Together, those agents complete most of the routine checks before the compliance officer even opens James’s file. By the time she reviews the application, the routine checks are already done. She only needs to focus on what requires human judgment.

Here’s the complete system

James’s application enters the system.

The orchestrator reads everything James has submitted and decides which specialist agents need to run. Four specialist agents then work at the same time, each checking one part of James’s application.

Their findings go to the risk engine. It applies the bank’s risk rules, assigns James’s risk level, and highlights the areas that need the compliance officer’s attention.

That output is then sent to an AI model. It turns the findings into a plain English summary for the compliance officer to review.

Every step is recorded so that the bank can trace every decision later.

Let’s look at how each part works, starting with the orchestrator.

Think about a good project manager.

A good project manager doesn’t assign every team member to every project. She first understands the work that needs to be done, then decides who should be involved.

The orchestrator does the same thing for James’s application.

It receives the application, reads through everything James submitted, and works out what needs to happen.

The orchestrator does three things.

First, it decides which agents should run.

The identity, screening and wealth agents run for every client, because a bank has to perform those checks on everyone it onboards.

The fourth check is the one that varies.

Some of James’s declared wealth comes from his business, so the orchestrator includes the business review agent. A client whose wealth comes only from a salary wouldn’t need it, and that agent would never run.

Second, it decides what information each agent needs.

The identity agent gets his name, nationality, residency and government ID. The screening agent gets his name, date of birth, and declaration that he may be a politically exposed person. The wealth agent gets his personal source of wealth, personal source of funds, and the supporting documents. The business agent gets the company details and supporting business documents.

Finally, it records why each agent is running.

In a bank, every check exists because a regulation requires it. That regulation shaped the check in the first place, and it’s what a compliance officer relies on when reviewing the case. So the orchestrator writes the regulatory requirement next to each instruction it gives to the individual agents.

So that months later, when a regulator asks why the screening agent ran on James, the reason is recorded in the trace alongside what the agent found.

In the prototype, LangGraph manages this orchestration. It takes the plan the orchestrator produced, runs the agents, and carries their findings forward to the next step.

Once that is in place, the specialist agents begin their work.

Each agent has one job

One verifies James’s identity. Another screens him against sanctions and PEP databases. A third reviews his wealth and source of funds. The last reviews the business he sold.

Because these checks don’t depend on each other, all four agents run at the same time. They don’t wait for one another to finish, which is one of the biggest reasons the onboarding process becomes much faster.

These agents follow a fixed set of rules rather than using an AI model. The same input always produces the same output. That’s a deliberate design choice, and you’ll see why when we get to the risk engine.

Now let’s see what each of those specialist agents does.

The Identity Verification agent

The Identity Verification agent starts with a simple question.

Is James really who he says he is?

A bank answers that by verifying his government-issued documents, such as a passport or driver’s licence, against external sources like Equifax or TransUnion.

We’ve kept it simpler in the prototype. The agent checks that his name, nationality and residency have been declared, and that a government ID has been uploaded. For James, all four are present, so the finding comes back as IDENTITY_VERIFIED.

The Screening agent

The Screening agent checks James against sanctions lists and politically exposed person registries.

A bank runs those checks through commercial providers such as World-Check, Dow Jones or ComplyAdvantage, which maintain the lists and keep them current.

In the prototype the lists live inside the agent itself. It matches James on name and date of birth, finds him in the PEP registry, and confirms he declared that status himself. The finding comes back as PEP_CONFIRMED.

The Wealth and Funds agent

The Wealth and Funds agent looks only at James’s personal money. Anything connected to his company goes to a different agent.

The agent answers two questions.

Where did this wealth come from? For James, it comes from his cryptocurrency holdings held through an exchange.

How is the money arriving? It is a one-million-dollar transfer from the same exchange account.

In a real bank, the second question would typically be investigated using a blockchain analytics tool such as Chainalysis. This helps determine the origin of the cryptocurrency before the funds are accepted.

The prototype performs a much simpler check. It verifies that James’s wealth documents, bank statements, and cryptocurrency exchange records have been provided. Since these documents are available, the agent returns the finding WEALTH_SUPPORTED_CRYPTO_PRESENT.

This finding does not mean that the cryptocurrency has been verified as legitimate. Instead, it indicates that cryptocurrency is involved and that the prototype cannot verify its origin. As a result, the case is referred to the compliance officer, with this limitation clearly identified.

The Business Review agent

The Business Review agent handles the other half of James’s wealth—the portion that came from selling his company.

To verify this, it answers three questions.

• Did James own the company?
• Did the sale take place?
• Does the amount he declared match the recorded sale value?

A bank answers these questions by comparing the sale documents with corporate registry records.

The prototype performs a simplified version of this check. It queries a simulated corporate registry, where James is listed as the founder and CEO of Whitmore Software Inc. It confirms that the sale took place and compares the declared sale value of eighteen million dollars with the registry record. The values match, so the agent returns the finding BUSINESS_SALE_SUPPORTED

The Risk Engine

Each agent has now returned its finding. Those findings flow into the risk engine.

The risk engine applies the bank’s internal risk rules to everything the specialist agents found.

In a real bank, the same system would do two things.

It would score the client on factors like where they live, what they do, and what kind of account they want. That score decides the risk level.

But some findings ignore the score entirely. A confirmed sanctions match stops the case, no matter what the score says.

The prototype applies the same kind of rules, but without the scoring. Every decision comes from a rule that either applies or doesn’t, and the scoring model is the part left out because it changes from bank to bank.

For James, two things stand out. His PEP status has been confirmed, and his funds are arriving from a crypto exchange where the origin of the crypto has not been established. Together, those two classify him as a HIGH RISK client.

The risk engine also tells the compliance officer what has already been verified and what still needs attention, so she knows exactly where to focus.

This entire step is rules-based rather than AI-based, and so are the specialist agents we saw earlier.

The reason is the same for both. A regulator can ask why James was classified as HIGH RISK, and the bank must give the same explanation every time. Rules guarantee that, because the same input always produces the same output. AI models are probabilistic, so they don’t.

So where does AI actually come in?

The risk engine produces a structured output that is easy for machines to read but not easy for a compliance officer to review quickly. An AI model takes that output and turns it into a plain English summary highlighting what has been verified and what still needs attention.

The summary never says “approve” or “reject”. The AI prepares the case, and the compliance officer makes the decision.

Even though James’s case is complex, the compliance officer doesn’t have to review everything from scratch. The routine checks are already done. She focuses on the two areas the risk engine flagged, reviews the supporting evidence, and makes her call. Approve, reject, request more information, or escalate to senior management.

Every step in this process is recorded. The bank can see which agents ran, what they found, how the risk engine reached its decision, and why the case was classified that way. In the prototype, LangSmith captures that complete trace.

So that’s how agentic AI works in banking

You started this article wondering why you need to read another article on agentic AI.

Now you know. Building agentic AI for a bank is different from building it for a technology company. Every decision needs a paper trail. A regulator can ask about any of those decisions at any time. And a human always needs to review what the AI has done before the bank acts on it.

James’s case shows how all of that works in practice. The orchestrator decides which agents to run. The agents check identity, sanctions, wealth, and business history at the same time. The risk engine applies the bank’s own risk rules and shows the compliance officer exactly where to focus. The AI model writes a summary in plain English. And every step along the way gets traced, recorded, and stored so the bank can explain the decision later.

That’s what agentic AI looks like when you build it for banking.