On Monday morning we successfully launched the Electronic Visa Waiver (EVW) service private beta in Kuwait to very little fanfare. The way a private beta should be launched.
The launch was combined with a user research field trip so we could test the system with these beta users.
I’ve been on many research trips in the past, but this user research trip was different: we were watching our users interact with our service in anger. This wasn’t a simulated scenario in a lab. It wasn’t a simulated scenario at someone’s home. This was for real.
It went smoothly. At the time of writing, 10 participants had made applications and 28 EVWs had been approved - for the individuals making the applications, and members of their families. The way we worked helped.
Planning
Like any research activity, planning is very important. It helps you identify what you want to get out of the research. It also helps you to anticipate things that might go wrong.
For our private beta in Kuwait we based our research plan on our previous user research trips to UAE and Kuwait.
We learned from previous difficulties. This time around we created guidelines for local translators to help, and being more aware of the culture, we scheduled the sessions for later in the day - between 12pm and 7pm) to make it more convenient for the users. We had iterated our user testing, and this made our launch easier.
We planned for things to go wrong - participants coming without the necessary documents, participants being delayed, technical issues.
When things went wrong, we were ready, we adapted and our users appreciated it.
Houston, can you hear me?
Communication with the participant
During the recruitment phase we put a lot of emphasis on explaining to the potential participants that they needed to have travel and accommodation booked in order to take part in our research.
Normally, you would not tell the user too much information in advance, but we decided to do this because this was a private beta launch - the user would be using the service for real.
On Thursday we were visited by Tobias Elwood, Minister at the Foreign and Commonwealth Office, and Matthew Lodge, the British Ambassador to Kuwait. Not a usual occurrence during a research session, you’ll agree.
The presence of high-ranking officials wasn’t ideal, as it put many of the participants on edge. It was a necessary part of the service launch, however, and when we explained the situation to the participants they felt more at ease.
Communication with the team
Communication between team members is important during any usability testing session. It’s even more important when your users are using your service for real, and your team members are 6,000km away.
During the live applications we kept in touch with our team in London via Slack. We alerted the team when the participant was about to submit the form so that the caseworkers would be ready.
This real-time communication is vital when a participant is sitting beside you in one country and the outcome of their actions is only seen in another.
It allowed us to identify issues, notify the necessary people and manage the participants’ expectations.
You’ve gotta roll with it
This user research trip was quite different from our normal work. It was a combination of user testing and service launch, meaning we needed to perform a complicated several role: part user researcher, part troubleshooter, part tester, and part childminder.
Some participants were mobile device users and preferred to use their smartphones, so we tested them using their phones.
As anticipated, some participants came without travel details, so we had sample travel arrangements prepared for them. In these cases we used those sessions as more formative research.
Some participants came with their children, so we kept them occupied while their parents took part in our research.
When something went wrong, we waited for a suitable moment, so we didn’t compromise the research, and then explained to the user what had happened. We then worked with them to find an answer, and ensured they left feeling satisfied.
What’s valuable is how we use what we have learned to inform the next service iteration.
We will use these learnings for next week when we continue with our research and continue to improve the service.
Let’s share
Do you have a different approach or thoughts on what we’re doing? We’re open to further improvements in how we approach digital transformation and would love to get your thoughts, inputs and experience in the Comments section below or via twitter
1 comment
Comment by Graham posted on
Great to see how you're conducting user testing in the field and adapting to your situation. Would be great to hear more how your findings shape the product in the future