Key Points
- Deep reinforcement learning is key to algorithms that can respond to a changing market dynamic.
- RBC is taking a client-first approach to AI development, ensuring its research is led by what clients really need.
- Future innovation will be key to maintaining a competitive edge in AI-enabled trading.
Just after the launch of RBC Aiden’s second AI-enabled algorithm, Arrival, Payments Processing and IT Services analyst Dan Perlin talked to two of the leaders behind RBC’s AI endeavours at the annual TIMT conference. Susom Ghosh, Head of Global Quantitative Sales Trading and Strategic Initiatives at RBC Capital Markets and Hasham Burhani, Co-Head of Algorithmic Research, have been on RBC’s AI journey from the start and know exactly how AI is likely to continue to change the world of trading.
Reinforcement vs supervised learning
When RBC Capital Markets teamed up with Borealis AI in 2016 to develop an algorithm for volume-weighted average price (VWAP), the team had no idea that it would be testing that algorithm in the midst of an unprecedented event – the pandemic.
When RBC Capital Markets teamed up with Borealis AI in 2016 to develop an algorithm for volume-weighted average price (VWAP).
“Every theory, explanation, expectation, everything that we had got put to the test,” said Ghosh.
This was where RBC’s early decision to use deep reinforcement learning as opposed to supervised learning really paid dividends. Supervised learning is where an AI model is trained on a given dataset. A typical example would be a classification problem – is this picture a cat or a dog? With supervised learning, the model would guess whether an image of a beagle was a dog and then the supervisor would say yes. If the model said a bulldog was a cat, the supervisor would say no, it’s a dog. This training goes on until the model has reached a decent level of confidence for new images.
But reinforcement learning is different, it’s similar to how human beings learn as children. Take a baby learning to walk for example. The baby doesn’t learn each step individually, he or she simply wants to go over there and fetch a toy. They flail around and figure out that they’re moving, then they roll over, eventually they crawl, then they stand up, then walk and then run. Each stage is successively better than the rest, so the reinforcement for the baby is that they’re getting better and better at getting over there.
RBC and Borealis AI used reinforcement learning to train their Aiden platform for VWAP because the outcome of the model is considered alongside the decision it makes.
“The live market is constantly changing. And so we thought, instead of fitting a model on a historical basis, can I build a model that can adapt in real-time to the actual market? So reinforcement learning gave us exactly that, and no other AI system would have been able to give us that.”- Hasham Burhani, Co-Head of Algorithmic Research, RBC Captial Markets
When it came time to test that model in 2019, they realized how important that flexible learning approach was.
“The biggest differentiator at that point was, there was no historical precedent,” said Ghosh. “So even if you were using supervised learning, what's your historical precedent to train an algo? Whereas here you had a platform that is essentially built to realize the changes in the market regime and then change its behavior accordingly.”
Client-first development
When creating the VWAP solution, RBC’s intent was to prove to clients that this technology is a paradigm shift. Following VWAP’s initial launch the team solicited client feedback to leverage the technology to solve for more complex challenges clients were facing i.e. solving for the Arrival Price benchmark, a key industry standard in trading.
“After launching Aiden, we went on a roadshow to talk to our clients and get their feedback and it showed they wanted to expand on VWAP… Arrival can essentially control multiple aspects of trading decisions. And on top of that, it has a more holistic approach to evaluating its own feedback, so it incorporates significantly more in terms of the trading experience, both when you’re trading and the impact after,” explained Burhani.
The next steps that clients want to take are in liquidity-seeking – one of the most popular strategies for clients, but also a complex challenge for AI – and expanding geographically.
“Largely, our success today has been in the US and Canadian markets. We actually just went live with VWAP, the original algo, in Latin America, Brazil and Mexico — and we are testing it in Europe. So I would say expanding this into other geographies and giving clients an advanced toolset for those markets are a pretty big ask,” said Ghosh.
Future innovations
Aiden is some way ahead of most competitors and the team wants to keep it that way by continuing to get feedback from clients and assess the big themes in AI. One of those is the incorporation of alternative data to gain an edge in training.
“When I imagine where we can go over the next little while, we’ll have the architecture to consume both structured and unstructured data, so that alternative data, which can vary in shape, size and representation, will be able to be used by the neural network architecture alongside the usual data – that will give you an edge going into the future,” Burhani said.
Ghosh added: “We’re also looking into other asset classes and other applications for this technology. And we're really hoping that in the next 12 to 18 months we're able to break ground on that and make some progress.”
