From the white board to the real world

How RBC Capital Markets is using artifical intelligence to help build the future of execution platforms

The traders at RBC Capital Markets have a new teammate. Don’t expect Aiden to join conference calls or working lunches, but the reinforcement-learning-powered algorithm—developed through an unlikely collaboration between AI scientists and traders—is delivering through its game-changing technology.

2017

It was nearing the end of the year, and, on RBC Capital Markets’ trading desk in Toronto, an eclectic team of AI scientists and equities specialists were poised to usher in a new era of trading. In order to help clients navigate ever-changing market conditions, they had developed an AI-powered trading platform over the past few years, choosing the gender-neutral name “Aiden” (with “AI” as the first two letters) to symbolize the partnership of human and machine. Through its AI brain (the central learner), Aiden would get smarter in real time, learning and improving with each trade.

Anticipation was high as the team prepared to send the first beta client order. Hasham Burhani, Lead AI Scientist on the equities desk and David Shi, an AI scientist, joined colleagues in the room and those who had called in over the phone to be part of “the moment”. A year’s worth of whiteboarding, developing, simulating, and refining between traders and academics, often late into the night and on weekends, would be tested by a single, live trade.

“Never in the history of RBC had so many people been looking at a single trade,” Steven Szeto, Head of Global Equity Execution Strategy, recalls. “Everyone was thinking: ‘Is Aiden going to trade well? Is it going to understand the market? Is this going to work?’ ”

Leah Hutton, David Shi, and Steven Szeto huddle in the AI lab on RBC’s trading floor in Toronto.

SUMMER-FALL 2016

A drive for innovation from the top

“Artificial intelligence is a transformational technology that is going to change our society over the next 50 years. Our vision, and our ambition at RBC, is to be the best in the world at unlocking the potential of this technology”

Dave McKay, RBC CEO

Just three years prior, Aiden was a glimmer in RBC CEO Dave McKay’s eye. A computer programming major in college who started his RBC career as a coder, McKay challenged RBC’s 84,000 employees worldwide to embrace the future.

“Artificial intelligence is a transformational technology that is going to change our society over the next 50 years,” McKay says. “Our vision and our ambition at RBC, is to be the best in the world at unlocking the potential of this technology to serve our clients, from the largest capital markets client to a retail customer, to create value and make their lives better.”

RBC employees across the bank took up McKay’s challenge. In equities research, Fardeen Khan, Head of Strategic Initiatives, started exploring ways data science—which includes AI as well as algorithmic data analysis and other sophisticated tools—could help analysts extract differentiated insights about companies. By September 2018, Khan had assembled a team of data scientists, now called RBC Elements. They collaborated with equity analysts to produce unique, compelling research that has made news and helped unlock powerful insights for clients.

On the trading desk, Co-head of Global Equities Execution Shary Mudassir and his colleagues had been working relentlessly with trading algorithms to execute better for clients. Mudassir, who had an MBA, even went back to school to get his master’s in AI.

At the same time, McKay was building a brain trust of AI talent. His first academic recruit was Dr. Foteini Agrafioti, who had invented a biometric ID based on heart rhythm and was named one of Canada’s “40 under 40” in 2017. Hired for the new position of  Chief Science Officer at RBC, Agrafioti, along with McKay, founded Borealis AI, a research institute dedicated to advancing artificial intelligence.

Offering the exciting opportunity to pursue pure, as well as applied, research and to publish in peer-reviewed journals, Agrafioti recruited machine-learning experts from around the world. The bank’s Canadian heritage was a plus: Canada is an AI power player, with the University of Alberta, in Edmonton, ranked third in the world for its computer science research.

Dave McKay greets employees at Borealis AI, RBC’s state-of-the-art AI Institute.

FALL 2016-SPRING 2017

A bold decision

Mudassir and Agrafioti were working on parallel tracks. Their paths converged one day in November 2016, when Mudassir tracked down Agrafioti and her team in their nondescript rented space at the University of Toronto.

“Our cultures were so different,” recalls Agrafioti. “On Shary’s side, they are very extroverted. On our side, some people wouldn’t even make eye contact. But we were amazed by each other.”

Before long, the two teams discovered that they could collaborate on game-changing technology to benefit clients. Agrafioti’s academics were researching reinforcement learning (RL), widely considered AI’s next frontier. Mudassir’s team was exploring how AI could help optimize execution in a dynamic market environment.

“We knew nothing about capital markets,” Agrafioti says. “But when Shary went to the whiteboard and explained electronic trading, we saw right away—yes, this could definitely be formulated as an RL problem.”

AI algorithms based on Supervised Learning (SL) rely on hundreds or thousands of “If A, then B” rules that coders write to enable the algo to make the best decision in as many scenarios as possible. But in a dynamic and often volatile environment such as capital markets, SL algorithms, while often highly successful, are limited by human imagination.

Reinforcement Learning (RL) algos, in contrast, rely on a feedback signal (reward) to optimize decisions in real-time. The “actor-critic” model (a state-of-the-art reinforcement learning technique) works in the following way. The algorithm takes an action. It then receives positive or negative feedback for that action based on how it performs relative to a benchmark; in the capital markets, that benchmark might be a metric of best price execution. With every new action, and every consequent reward or penalty, the RL algo gets “smarter” about its environment, and, in theory, continues to learn without limits.

The potential for RL to give human traders ever-improving intelligence in a dynamic market was the animating idea behind Aiden. “We thought, ‘What if we could design an RL algo that listened to the market directly, that could change its own behavior without the need for programs that constantly change lines of code?’” says Hasham Burhani, Lead AI Scientist on the RBC equities desk. “We could then be much more responsive and efficient for our clients.”

With the support of McKay and RBC Capital Markets leadership, Mudassir and Agrafioti aligned on a vision. Instead of improving the bank’s existing trading algos, they would do something entirely new and take a bold leap into RL. The goal: to build a next-gen trading algorithm to establish RBC’s leadership in AI trading—and set a new standard of execution excellence for clients.

“We didn’t want to build a better combustion engine by fine-tuning various parts,” Mudassir says. “We wanted to build an electric car.”

SPRING 2017-SUMMER 2017

Creating Aiden and Supporting Ethical AI

The team started putting the pieces together. AI experts Burhani and Shi set to work understanding the complex dynamics of equity trading, while equities pro Szeto got up to speed on what RL could do. In just under one year, Burhani and Shi had written the first draft of the code for the algorithm that would eventually become Aiden.

“Working with AI scientists who think very differently from a traditional quantitative analyst such as myself was extremely interesting,” says Jas Sandhu, Head of Algorithmic Trading for Equities. “To build a new system, we needed something like a neural network that could scale with more data and more inputs.”

Jas Sandhu
Jas Sandhu
Rich Steiner monitors client activity on RBC’s trading floor in New York City.

Throughout development, the team always had one thought in the back of their minds: Any new technology, none more so than AI, would raise questions around explainability.

To that end, the team took a client-centric approach in defining Aiden’s features and benefits, similar to the approach RBC took when developing THOR®, a smart-routing technology that helps address the problem of disappearing liquidity and assists clients with competing with High Frequency Traders. It was important to help clients feel confident about how Aiden works in the market, so Aiden was designed to operate within an extensive risk management framework. Additionally, Borealis’ scientific focus on explainability, on being able to explain the actions taken by AI algos, meant that Aiden could ultimately provide significant benefit in this area.

Today, RBC continues to help drive the conversation around socially responsible AI, from McKay’s public statements on the subject to RBC’s support of ongoing research into ethical AI practices.

SUMMER-FALL 2017

In the lab with Aiden

With the Aiden VWAP algo written, there was only one way to see if it would actually work: trade in live markets. Because a RL algo learns in real time, it requires multiple experiences to make good and bad decisions, receive the positive and negative feedback for those actions, get smarter, have more experiences, learn more, and so on. But, obviously, RBC wasn’t willing to “train” Aiden with clients’ money, or even, at first, its own.

“You need a lot of time and experience before an RL algo gets better,” Szeto says. “But you can’t do that in live markets where billions of dollars are at risk.”

In true RBC fashion, innovation and collaboration led to a plan. Szeto and Burhani called on the Capital Markets IT team, which wrote a program to simulate a live market environment. This in itself was an ambitious task: The simulator needed to reflect the complexity of real markets, including both buyers and sellers with differing objectives and fluctuating levels of liquidity throughout the trading day. After just three months, the simulated exchange environment was up and running, enabling Aiden to “trade” and get feedback on its decisions in real time with no money changing hands.

Steven Szeto, Hasham Burhani and Selina Wang gather to see how Aiden performs in the simulated environment.

Late 2017

From lab to live

After a successful run in a simulated environment, it was time to test Aiden further. While much of North America was enjoying downtime three days after Christmas, the Aiden team huddled around a desk in Toronto, preparing to watch Aiden execute its first trade on a live exchange, using  the bank’s own capital.

The group kept a close eye on Aiden’s movements in the hours following the initial trade, inviting others from the trading desk to stop by to see the algorithm working in live markets. At the end of the day, the results were good, with promising slippage numbers from its core benchmark.

Over about a month, further testing determined that the algorithm showed early signs of success. At this point Hasham and the team decided to stop phase one of the testing and begin to rewrite parts of the platform and prepare for the next phase of expanded testing.

“Aiden didn’t yet fully understand what it needed to do,” Szeto says. “We knew it had more potential.”

So, the determined team was back to the whiteboard, pushing one another to help Aiden get even better. The more they experimented and refined, the more confident the AI experts became.

Hasham Burhani and team go back to the whiteboard as Aiden continues to learn and improve.

Early 2018

Going Live - Again

In early 2018, the next phase of testing began. This time the first order to be sent was by Szeto on the desk.

The room grew silent as he hit a few keystrokes. Once again using RBC's own capital, Szeto started Aiden with two small orders—$20,000 each. It then followed up with two large orders of $900,000 each.

By the end of the day, it was clear—the test was an unqualified success. High fives were shared by the team, with excitement high on the back of what was achieved thus far. In the weeks and months that followed, Aiden would continue to learn and improve.

Susom (Sush) Ghosh watches moments before Aiden's first test orders are placed for the US platform.

2020

Aiden Launches Globally

With the approval of Bobby Grubert and Ahmed Kachenoura, co-heads of RBC Capital Markets’ Global Equities business, it was time to move to the next level of validation: Partnering with a few trusted clients who were eager to see how RBC’s traders could put Aiden to work on their behalf.

Today, after years of collaboration, testing, tweaking and testing again, Aiden is live. Even now, RBC’s leaders are looking ahead, as its AI scientists and trading experts investigate new ways to bring the power of advanced machine learning to RBC clients worldwide.

“After just four years, we’ve built an innovative product like Aiden in partnership with our employees and clients—and it’s solving real problems in a material way,” McKay says. “We’re incredibly proud of where we are today, and excited about where we’re going in the years ahead.”

Aiden logo
Proud Members of Team Aiden in Toronto.