pepper canada toronto hsbc

HSBC Bank Canada’s new Client Experience Ambassador at the bank’s 70 York Street branch. HSBC is the first major bank in Canada to deploy Pepper. (CNW Group/HSBC Bank Canada)

August 28, 2019      

Stroll into an HSBC Bank branch office in New York, Seattle, Beverly Hills, or Toronto (see below), and you’re likely to encounter a 4-foot-tall, all-white, 60-pound, semi-humanoid robot that’s clearly trying to get your attention by using the same verbal and nonverbal cues that we’re all familiar with. As you approach Pepper, developed by SoftBank Robotics, it might make a humming noise to indicate that it has verbal abilities and wants to start a conversation. It sways back and forth to display a friendly, welcoming sort of animation. It might do a little dance. And it nods its head and moves its arms to approximate human body language.

Once you get closer, Pepper’s facial recognition and social skills kick in. Pepper looks right at you. A pale greenish pulsating glow accentuates Pepper’s round black eyes that sit recessed in its head. Pepper speaks: “Welcome to HSBC. How can I help you today?” And that’s where the fun starts.

Studying real-world interactions

Matt Willis, Ph.D., design and HRI lead at SoftBank, has been intently studying the real-world interactions between Pepper and bank customers in order to continue to improve the effectiveness of the robot. He says initial reactions to Pepper run the gamut; some people stop and watch from afar, some run right up, some just wave, some start talking, some try to shake hands, and many whip out their phones to take selfies and videos.

HSBC Pepper Beverly Hills

“Pepper” at HSBC Bank in Beverly Hills, California on March 6, 2019. (Photo by Amanda C. Edwards / @acefotopro)

You can always tell who has Alexa in their house, because those customers ask the robot for its name and then use “Pepper” as the trigger word, even though that’s not necessary. Willis says that when it comes to human-robot interactions, there’s a learning curve on both sides.

Matt Willis SoftBank Robotics

Matt Willis, SoftBank Robotics

The design challenges that Willis faces are far less about the technological aspects of building a robot than about the psychological, behavioral, linguistic and social dimensions of the human-robot experience. And Pepper has a very specific use case: The goal is not to just be entertaining; the goal is to provide real business value.

At HSBC, Pepper’s role is to dispense information about banking products and services, and to answer lower-level customer queries. When the situation calls for it, Pepper can refer customers to the appropriate bank employee. In other words, Pepper serves as a sort of walking, talking, interactive information kiosk.

Non-threatening, yet intelligent

One of the key design criteria was making sure that Pepper was not imposing, threatening or off-putting in its physical appearance, particularly for a public setting like a bank where visitors can run the gamut from children to the elderly. That’s why Pepper is short, has a soft, rounded shape and a childlike appearance.

In addition, there was no attempt to make Pepper look human. “Pepper is a robot and doesn’t pretend to be more than a robot,” Willis says. “Pepper is not trying to be human – it’s trying to be social.” This approach avoids the whole ‘uncanny valley’ issue that other humanoid robots that look “too human” could face.

Pepper at Robotics & AI Summit

Pepper is designed not to be pushy or invade humans’ personal space. Image: Matt Wagner

In addition, the team worked on Pepper’s voice to make it different from the halting, monotone, and usually female voice that humans hear when receiving a robocall or encounter a call center bot.  Pepper is different. A team in Paris developed a unique voice that matches Pepper’s physical appearance; in other words, Pepper has a clearly non-human voice that is meant to be friendly and to show emotions, particularly enthusiasm and excitement. At the same time, the voice isn’t too cartoony, because it has to convey to the customer that Pepper has intelligence, and is capable of discussing serious topics like banking products and services.

Gesture and movement are additional challenges, says Willis. Pepper needs to find the right balance between attracting customers and not being pushy. So, once Pepper grabs someone’s attention, it doesn’t invade anyone’s personal space; it’s up the customer to approach the robot.

In addition, the robot’s gestures and hand movements need to take into account all of the potential misunderstandings that occur in conversations. For example, if a customer asks a question and Pepper nods, does it mean that Pepper understood the question, or does it mean that Pepper is answering ‘Yes’? Willis said these are the subtleties that his team is constantly refining and improving.

Natural language barriers

If you think about the times when human conversation results in some sort of confusion or miscommunication, you can understand the challenge with robot communication. The last thing the SoftBank team wanted was a customer getting frustrated or exasperated with Pepper.

To address this, Pepper was designed to be bi-modal. It has a touch-screen tablet attached to its chest that displays several options for customers, who have the choice of interacting with Pepper verbally, via the tablet or switching seamlessly back and forth. For example, Pepper might point to the tablet and say, “You can select one of these options or just ask me a question.”

Willis says he tapped into the expertise of linguists to help Pepper structure sentences in a conversational manner, and to vary its responses so it doesn’t sound too repetitive. Other challenges include what to do when Pepper doesn’t understand a question, or understands the question but doesn’t have an answer. Pepper might say something like, “I don’t have an answer to that, but here are some other bank services that you might be interested in.” Or Pepper might call for back-up in the form of a human bank employee.

Results and the future

While people are concerned that robots might take away their jobs, Pepper is generating so much foot traffic in HSBC branches that the bank is actually boosting headcount. HSBC reports that Pepper is driving an increase in ATM transaction volumes and a rise in new credit card applications. One other important caveat: Pepper does not collect any customer data or personal banking information; it simply provides generic information on bank products and services.

Willis says the possibilities are endless as people become more accustomed to humanoid robots, and as Pepper gets better at understanding context, tone of voice, facial expressions, and body language.

For example, in a retail setting, Pepper might answer questions on where to find a certain item. But an even more sophisticated solution might link Pepper to real-time inventory levels, so Pepper can also tell a customer that a particular item is out of stock before the customer tries to find the item on their own. And in the banking scenario, Pepper’s facial recognition technology could be linked to a customer database, so that Pepper can welcome repeat customers by name and offer them personalized services.

Willis sums up his mission this way: “The challenge is understanding how we can develop robots that bring value to the world.”

Pepper heads to Canada

pepper canada toronto hsbc

HSBC Bank Canada’s new Client Experience Ambassador at the bank’s 70 York Street branch. HSBC is the first major bank in Canada to deploy Pepper. (CNW Group/HSBC Bank Canada)

HSBC Bank Canada today announced that Pepper will be deployed as its newly created “Client Experience Ambassador” role at a branch on 70 York Street in Toronto.

“I am very excited about Pepper and I know that it will get others more excited about coming into the bank,” said Larry Tomei, EVP and head of retail banking and wealth management at HSBC Bank Canada. “When talking with our customers, Pepper will help educate them to get the most out of our products and services and learn about other options. And when you meet Pepper, you can’t help but smile!”

Like Pepper’s functions at other HSBC branches in the U.S., Pepper will greet customers, ask them what help they need, and invite them to explore products such as credit cards or other services. Customers can pose with Pepper for selfies and upload them to social media using the hashtag #MeetPepper.

The bank said it has been making significant, ongoing investments in digital channels to respond to customers’ needs. In May 2019, it opened its Global Data & Innovation Lab in Toronto, which includes an artificial intelligence ecosystem that uses advanced machine learning techniques to analyze up to 10 petabytes of data from 1.6 million corporate and institutional clients.