With data becoming a core business asset for financial companies, artificial intelligence and machine learning (AI/ML) continues to be a focal point in maximising the competitive advantage of this ‘new oil’.

This is according to a new study by LSEG Labs, The Defining Moment for Data Scientists, on the applications of AI/ML in financial services.

The study found that the adoption of AI/ML within organisations has remained steady since 2018. Over the last three years, between 40-50% of respondents reported deploying AI/ML in multiple areas.

The survey is based on responses from 482 data scientists, quants, model governance professionals and C-suite executives, from both sell-side and buy-side financial institutions.

The report noted that companies are now leveraging both structured as well as unstructured data, and that the use of alternative data is also on the rise. In this context, natural language processing (NLP) and deep learning models are both gaining popularity, while firms switch to a more modular approach for AI/ML and cloud integrations.

Microsoft leads the way in the NLP race for now

While just 2% of respondents reported working with unstructured data in 2018, this figure has now grown to 19% in 2021. And with this development, NLP has come to the fore.

Cloud-based off-the-shelf text analysis was especially in demand, with 60% of respondents relying on Microsoft for text analysis. Microsoft was closely tailed by big tech competitors Google (54%) and Amazon (31%).

The competition is stiff but Microsoft has had some wins. Its Turing universal language representation model, which trains 100x faster than its older models, currently ranks second on Google’s XTREME index (a benchmark for cross-lingual transfer learning). Elsewhere, the company’s AI translator now translates 103 languages, closely matching Google’s 108 languages.

But apart from its NLP services, Microsoft has also emerged ahead of the pack as a cloud provider because, as the LSEG Labs report noted, respondents were more likely to use NLP services from the same company that provided the cloud service they relied on.

In this case, over half of the survey’s respondents (55%) reported using Microsoft’s cloud services, as compared to Amazon (48%) or Google (38%).

That’s not to say that companies are not looking at open source libraries. The report found that Flair, NLTK and spaCy were the most popular libraries for analysing and processing unstructured data. Meanwhile, a tenth of respondents used open source tool Hugging Face to experiment with large, pre-trained models.

Deep learning emerges as preferred training type

Besides NLP, deep learning has also emerged as a top trend in the application of AI/ML, with supervised learning now taking a backseat.

84% of respondents reported using deep learning models, versus 66% for supervised learning, 54% for unsupervised learning, and 46% for reinforcement learning. Transfer learning was the least popular training method, with just 15% of respondents opting for it.

The LSEG Labs report attributed this surge in adoption of deep learning to its commercialisation, through third-party APIs as well as off-the-shelf solutions. It also attributed the increased adoption to increased NLP applications.

Further, one in six respondents expected that deep learning would continue to have precedence in 2022.

Interoperability is key to building the right AI/ML capabilities

Companies are increasingly looking at outsourced, interoperable AI/ML solutions from a strategic perspective, the report noted.

Most companies (72%) preferred third-party integrations with their internal applications, while 47% opted for open source options. Just 26% were keen on buying full solutions from vendors.

This was subjective to the size of the company, however. Those with over 100 data scientists were more likely to choose a full solution provided externally, while in firms with less than 25 data scientists, integrating third-party vendor solutions into their own applications was more popular.

In choosing a tool, it was important for firms to take a “modular” approach, to create solutions that are unique to their needs. This means firms will need to “ build a mindset of being multi-platform,” and leverage interoperable and transparent data capabilities.

Concerns around model governance and tech

The LSEG Labs report noted that as data science teams within organisations grew, so did the need for model governance frameworks. Model governance was particularly prioritised by organisations with a team of 51 or more data scientists.

Regulation largely influences organisational outlook towards model governance, the report showed. Investment in model governance is expected to stem mainly from regulatory considerations (51%), while cost and quality concerns (49%), and ethical standards (42%) were also salient.

The report added that European Union regulations such as the General Data Protection Regulation, and new guidelines for excellence and trust in AI, are reshaping the way companies work in the intersection of data and AI/ML. Similar regulations also exist in the APAC region, such as the Personal Data Protection Act in Singapore, or the Personal Data (Privacy) Ordinance in Hong Kong.

Meanwhile, although stakeholder trust in the applications of AI/ML to data is a concern, firms are more inclined to implement model governance models for better statistical performance and explainability. This may have repercussions going forward, as the report points out that “ethical and bias issues are likely to emerge” as less-explainable AI/ ML frameworks such as deep learning, bias-laden data such as unstructured data, grow in adoption.