Your feature has now been built.
It has passed testing in both lower and higher environments. From a technology perspective, everything is working fine.
But there is still one important step left. The feature has not been tested by real clients.
That is what pilot testing is for.
What’s pilot testing?
Pilot testing means releasing a pilot version of the app to a small group of users before it is rolled out to all 1 million high-net-worth clients.
These users interact with the feature exactly as they would after launch. They call their relationship managers through the feature, send emails, and see what happens when the relationship manager is unavailable. At the same time, the relationship managers and backup support teams receive those requests exactly as they would once the feature goes live.
Pilot users test the scenarios outlined earlier in the pre-launch plan. You want them to test all the scenarios so that nothing is missed.
Invite clients from your key segments
You invite high-net-worth clients from your main client segments.
The most important segments are retired business owners and passive heirs, since these two groups represent roughly 70% of the high-net-worth client base.
If possible, you can also invite clients from other segments, such as entrepreneurs and professionals.
Getting feedback from different segments helps you understand how different types of clients react to the feature before it goes live to the entire client base.
The challenge of recruiting pilot participants
Recruiting high-net-worth clients for pilot testing is not always easy. These clients are extremely wealthy, busy, and often have limited time.
The best way to invite participants is through relationship managers. Relationship managers know which clients have strong relationships with the bank and may be open to participating in the pilot.
If it is hard to recruit high-net-worth clients for pilot testing, another alternative is to invite internal employees who are also high-net-worth clients of the bank. These internal clients are usually senior managers or executives. They are easier to coordinate with and can still provide meaningful feedback.
What happens during pilot testing
Pilot testing often takes place at the bank’s office.
Pilot users receive devices with the pilot version installed, or they install the pilot version on their own phones. They then begin testing the feature while the product team observes how they use it.
Whenever possible, you should sit in the room while pilot users test the feature so you can see how they interact with it. Watch how they navigate the screen, where they tap, and where they hesitate. These observations often reveal usability issues that would be difficult to detect through analytics alone.
Analytics can show what actions users take, but watching users directly helps you understand why they behave the way they do.
For example, a client may hesitate before tapping the contact button or spend time trying to understand what the feature does. These signals help you identify usability issues before the feature goes live to all clients.
Asking for feedback
After the testing session, ask a few simple questions.
What do you think about this feature? Was anything confusing? What would make this better? Would you use this when traveling?
You are collecting qualitative feedback from real users who represent a sample of your audience. This feedback is extremely valuable. You document the key observations and then send a short update to leadership summarizing what you learned during the pilot phase.
This is only the first round of feedback. Once the feature goes live, you will collect feedback from a much larger group of clients.
When to move forward
Pilot testing uses only a small number of users, so the goal is not to collect extensive feedback. The goal is to confirm that the feature solves the core problem.
In this case, the problem was simple. Clients could not reach their relationship managers when the relationship managers were unavailable.
During the pilot, you verify that clients can successfully contact their relationship managers using the feature and that the backup team supports the client when the relationship manager is unavailable.
Once you confirm this, the feature is launched to high-net-worth clients in phases. It first goes live to 10% of the one million clients, followed by 25%, then 50%, and finally 100%.
What have you learned?
Pilot testing confirms that the feature works with real users before it reaches the entire client base.
You invited clients from your key segments, observed how they interacted with the feature, and collected early feedback. Most importantly, you confirmed that clients could reach their relationship managers through the feature when they needed assistance, and that the backup team responded promptly when the relationship manager was unavailable.
Once pilot testing is complete, the feature moves to the next stage: launching to high-net-worth clients in phases.
At that stage, you begin monitoring adoption, response times, and business impact.