Articles related to AI and machine learning algorithms.

Machine Learning in Finance

/in /by

It’s known that almost all industries are influenced or about to be influenced by the appliance of Artificial Intelligence. Perhaps operational efficiency is what makes Artificial Intelligence so attractive for business owners across different sectors. Operational efficiency could lead to the reduction of costs, increased performance, speed up some processes or increase the quality of services.

In this article, we would like to cover the appliance of Machine Learning across the financial industry by presenting interesting use cases and examples to structure this content. 

Artificial Intelligence is assisting financial institutions to drive new efficiencies and deliver new kinds of value. Autonomous Research predicts that Artificial Intelligence will represent $1 trillion in projected cost savings for the banking and financial services industry. By 2030, traditional financial institutions will save 22% of costs.

Let’s get something straight here and let’s define Machine Learning vs Artificial Intelligence. These two terms are always used side by side of each other, but they are different. With different, we mean that Machine Learning is a subset of Artificial Intelligence. Artificial Intelligence refers to create intelligent machines. Machine Learning refers to a system that can learn from experience. In this article, we may mention both but with the given simplistic definition you already know what we are referring to.

Machine Learning layers

Source

Challenges of applying Artificial Intelligence in finance

Before jumping into the use cases that we have gathered for this article, let’s take a look at the most common challenges of applying Artificial Intelligence in firms within the financial industry. 

As previously mentioned costs and budgeting required to automate some of the processes in finance could be one of the main important challenges financial firms face. Additionally, regulatory requirements sometimes could be also a burden. They are complex frameworks and the required research phase could be time-consuming and hiring regulatory consultants could be costly.

Perhaps another challenge these firms face is the lack of structured or sufficient data to process and train their data and test if models are efficient enough. Adding to that lack of in-house skills and knowledge as well as missing the development environment (lab) that data scientists can join and apply approaches of AI. 

Another challenge could also come from market maturity and retail readiness to use Artificial Intelligence-powered tools.  

Case 1. Fraud detection 

One of the very important appliances of Machine Learning in finance is fraud detection. With the advent of instant payment and global transfer services, the volume of payments and transfers has dramatically increased. So is a notable amount of transfers that aren’t with good intentions including money laundering. The estimated amount of money laundered globally in one year is 2-5% of global GDP, or $800 billion to $2trillion.  

An advantage that Machine Learning has brought to fraud detection is the amount of data that can be processed by machines with minimum or zero human intervention. The appliance comes with more accuracy in the detection of fraudulent activities. 

For instance, the credit card fraud detection problem includes modeling past credit card transactions with the knowledge of the ones that turned out to be a fraud. This model is then used to identify whether a new transaction is fraudulent or not. The aim here is to detect fraudulent transactions while minimizing incorrect fraud classifications. Anomaly detection is a commonly used model for the credit card fraud detection problem, this is a technique used to identify unusual patterns that do not conform to expected behavior, called outliers. Anomaly detection or pattern recognition algorithms start with creating processes that find the hidden correlation between each user behavior and classify the likelihood of fraudulent activity.  

Discovering hidden and indirect correlations can be named as advantages that Machine Learning based algorithms bring compare to basic Rule-based fraud detection algorithms. Machine Learning-based algorithms also reduce the number of verification measures since the intelligent algorithms fit with the behavior analytics of users. A more automated approach to detection of fraud is also another asset for Fraud Detection algorithms applied by Machine Learning since they require less manual work to enumerate all possible detection rules. 

Case 2. Know Your Customer (KYC)

Improving the KYC process is one of the operational efficiency of artificial intelligence and machine learning algorithms that have been brought to the financial and banking industry. The appliance of Machine Learning on the KYC process is mostly implemented by traditional banks and neobanks. The main reason is the continuous evolvement of requirements from regulators. The due diligence required on customer registration that is required by regulators in banking is broad and complex. Machine Learning due diligence modules can be utilized to create robust automation and improve the process of KYC for institutions that are aiming to have an efficient retail onboarding. This will decrease the human intervention needed during the onboarding process and increases the accuracy as well as reducing the costs. 

Again neobanks are streamlining the KYC process with enhanced user interface and user experience. Simplifying and automating the KYC process can reduce the cost of onboarding and customer application process by 40% (source Thomson Reuters). 

One of Machine learning techniques used in the KYC process is the Facial Similarity check, which is to verify that the face in the picture is the same with that on the submitted document provided e.g. Identity Card. The customer will only be verified and pass the KYC process if the results of both Document and Facial Similarity checks are ‘clear’. If the result of any check is not ‘clear’, the customer has to submit all the photos again.

Case 3. Algorithmic Trading

The algorithmic trading with a technological infrastructure brought many advantages to the trading world e.g. the ability to trade in under a millisecond with the best prices available or the ability to simultaneously monitor and trade across multiple exchanges, and all with reducing the human error from trading. Algorithmic trading constitutes 50-70% of the equity market trades and 60% of futures trades in developed markets. 

Many hedge funds started to utilize Artificial Intelligence within the algorithmic trading world. It’s understandable that most of them do not disclose the details and mechanism of their approaches in applying Artificial Intelligence in their trading algorithms, but it’s understood that they use methods of Machine Learning and Deep Learning. There is also a wide appliance of sentiment analysis on the market in which the result can be used in trading. The main objective of applying sentiment algorithms is to obtain knowledge about the psychology of the market. 

Machine Learning is assisting the trading industry in order to leverage the market with fundamental and alternative data in order to research alpha factors. Supervised, unsupervised and reinforcement learning models are being utilized to enhance the processing of algorithmic trading strategies. Methods can be applied to optimize portfolio risk calculations and further improve the performance of the portfolios. 

Deep Learning models also have been widely applied in trading. Deep learning models with multiple layers have shown as a promising architecture that can be more suitable for predicting financial time series data. In a tested practice, the algorithm trains 5-layer Deep Learning Network on high-frequency data of Apple’s stock price, and their trading strategy based on the Deep Learning produces 81% successful trade and a 66% of directional accuracy.

Case 4. Chatbots and customer support

Reducing customer churn is perhaps one of the main criteria of financial institutions and banks. Generally, customers and especially millennials ones for neobanks do care about the customer service and support they receive. Chatbots and instant messaging apps could potentially increase communication quality between business and its customers. According to research by Juniper by the year 2023, the use of chatbots can reduce the operational costs for banking, retail, and healthcare business sector by $11 billion. 

The advantages that chatbots can bring to the industry are definitely increasing customer satisfaction and customer engagement rates. The speed of action and processing of many inquiries and threads at the same time could also be mentioned as a big advantage of chatbots and messaging systems. 

There are four types that Chatbots could be classified. Goal-based Chatbots, are designed for a particular task and set up to have a short conversation in order to complete a task given to them by a user. Perhaps this is the most common Chatbot. Goal-based Chatbots are deployed on websites to help visitors to answer their questions during their visit. 

The second type is knowledge-based Chatbots, these Chatbots use the underlying data sources or the amount of data they are trained on. Such data sources could be open-domain or cloud domain data. Usually, Knowledge-based Chatbots answer questions by providing the data and source of that data. 

The third category of Chatbots is Serviced-based. Such Chatbots are classified based on facilities provided to the customer. It could be personal or commercial information. Users of such Chatbots could place an order of a commercial good via the Chatbot. 

The fourth category is Response-generated Chatbots. Response Generated-based chatbots are developed based on what action they perform in response generation. The response models take input and output in natural language text. The dialogue manager is responsible for combining response models together. To generate a response, the dialogue manager follows three steps.

Types-of-chatbots

Source

Case 5. Automated Wealth Management with Robo Advisors

Wealth management is an industry and operation costs are a large burden for some of the firms. As wealth being transferred from one generation to a more tech-savvy one and considering the millennials are in their prime earning and spending years, the presence of automated and entirely digital investment advice tools can be expected. It’s expected that by the year 2022 the Robo Advisors revenue might reach 25 billion that is up from $1.7 billion from 2017, considering these tools are relatively cheaper to how the investment advice is being delivered traditionally, from 3% to 5% of assets managed to digital ones with 0.25% to 0.75%. 

There are practiced used cases where the processes of asset allocation modeling, portfolio construction, and optimization, as well as a portfolio recommendation systems, were bundled with Machine Learning techniques in order to enhance the current approaches. 

One example is the portfolio recommendation system that was designed to be implemented on top of a Robo Advisor and be utilized with the mean-variance optimization method was implemented using weighted linear regression. The model shows that adding a portfolio layer on top of the stock regression results is increasing the success rate (profit accuracy) up to 86.69% when success is calculated by the profitability of the recommendations. Moreover, it helps to reduce the risk by distributing the budget over a set of stocks and tries to minimize the reflection of the regression errors to the profit.

Robo Advisors come with many notable advantages, such a complete, online and real-time reporting dashboard to customers which can be checked on the go with mobile apps and dashboards. 

As previously mentioned they reduce the costs of operations for firms providing investment advice service and the fees that are clients charged. Robo Advisors are fully digital and they have online onboarding for clients which leads to expansion of client base for firms.

When can you apply AI (is your firm ready?)

There are few aspects to which we could measure the readiness of a firm to utilize Artificial Intelligence and Machine learning into their processes. A solid technological infrastructure is the most important element. An infrastructure that is put together to manage the whole lifecycle of data, from getting to cleaning to processing and feeding algorithms. The availability of the data can not be stressed more. 

The regulatory compliance as mentioned in some of the cases above e.g. in KYC processes is a crucial process to be taken care of before applying Artificial Intelligence into processes. Audit trails, transparency, result supervision, and reporting mechanisms are some of the high-level requirements from financial authorities. 

Talents as resources from data engineers to data scientists specialized and familiar with financial processes is another important criteria before kick-starting with Artificial Intelligence projects. Their ability to understand the sector and ways to improve it should be taken into account in their hiring process. Eventually, they need to start training the existing data with an accuracy level as a requirement for the models used. 

How to start your machine learning project?

  • Start with a question 

Before anything starts you need to start with the question, what is it that we want to improve with our Machine Learning algorithms? This should specify and clarify the objective of the project. 

  • Understand your data

Not every question can be answered with any data. You need to have the right data for the right question. This is practiced by receiving, cleaning and processing data. Running exploratory analysis on your data and making sense of some of the summaries obtained could be the initial stage to which you will know that if your data has the potential to answer your questions. 

  • Modeling

Once you found clues in your data associated with your question it’s time to try to write algorithms to find patterns that leads to successful or unsuccessful journeys. Usually, data scientists do this by fitting the most suitable Machine Learning models into the data, find correlative and statistically significant patterns and try to test the accuracy. 

  • Evaluation

In the modeling section, we talked about training your data and once we have found the best models that suit the question, the answer and the data and now its time to evaluate, in other words, test your models. Data scientists will keep on testing the models with new data to see if their models do not only work for one dataset. 

  • Deployment

Once the fitting algorithms are certain that works, it’s time to deploy. Generally, this means deploying a code representation of the model into an operating system to score or categorize new unseen data as it arises and to create a mechanism for the use of that new information in the solution of the original business problem.  Importantly, the code representation must also include all the data preparation steps leading up to modeling so that the model will treat new raw data in the same manner as during model development.

Schedule your appointment right now to learn more

Contact us

Market sentiment analysis

/in /by

What is market sentiment analysis? 

Also known as opinion mining or emotion artificial intelligence, or textual sentiment analysis aims to process and extract the subjective content of descriptive and written work. This objective is achieved through the analysis of the source’s opinions, or their evaluation towards a topic or a product as well as sentiments, attitudes, and emotions. 

In trading, investment banks and hedge funds are trying to take advantage of sentiments of the market to help them make better predictions about the financial market. Some of the very accomplished firms including DE Shaw, Two Sigma and Renaissance Technologies have been reported to use sentiment signals. In some of these approaches, sentiment signals are blended in with other data such as transactional data (prices, historical returns or dividends). Sentiment analysis can be performed as stand-alone research prior to traditional trading or it could be equipped into algorithmic trading. 

Techniques such as Natural Language Processing and Text Mining are employed in order to perform analysis and the extraction process. 

Types of sentiment

Based on the typology proposed by Scherer, there are two types of sentiment while performing sentiment analysis. One type is the “attitude”, which is a narrow definition of sentiment, whether the opinion is “positive” or “negative”.

The second type of sentiment within sentiment analysis is the “emotion”. This indicates the eight “basic emotions”, which are in four opposing pairs, joy-sadness, anger-fear, trust-disgust and anticipation-surprise. 

Steps to perform sentiment analysis on market data

Step one: data collection

In order to perform sentiment analysis on market instruments e.g. a stock, there first should be the source to which the data stream flows for sentiment algorithms to start processing. Such data sources are best to be provided through APIs to reach automation in the collection process. One common and widely used source of data for stock market sentiment analysis is Twitter. Through the search query API provided by Twitter, the thrid-party system can query data via the REST API with given conditions such as location, language and time and etc. 

Step two: Classifier engine

Once required corpus data created for the sentiment analysis, there should be a step to extract features from the data. Different domains should design classifiers differently since words could mean differently across different industries. Having said that, most of the sentimental words (basically adjectives) are self-explanatory. Some of the methods used to create word classifiers and feature extraction engines are Naive base, Support Vector Machine (SVM) or K-clustering methods. 

Step three: predictive modeling

Once the classifier engine is in place, the data scientists will start the process of examining the best predictive models for the data. Which makes this step into two phases, the training, and testing (predicting). during the training phase, the data scientists will train different models and fine-tune them in order to use it for the second phase which is testing which will result in obtaining an accuracy level. This process iterates until there is an acceptable accuracy level determined by domain experts and data scientists. Results of such tests should also be checked if whether they are statistically reproducible or not, basically if the model achieves a similar accuracy level on different datasets throughout time.

Are price and sentiment correlated?

There are numerous researches conducted over this question, nevertheless, the general intuition in the market is that; there is a correlation between sentiment and price in the market. In experimental work with consideration to asses whether the financial market reacts to relevant news events, the sentiment analysis was performed using both a standard model and an enhanced temporal model. Within the temporal test, they associate the sentiments with the corresponding temporal orientations by classifying each sentence with one of the four temporal categories (past/present/future/unknown) and calculate the sentiment strength accordingly.

 

Correlation-market-sentiment-price

The result of this practice concluded that the casualty test experiment, two competing hypotheses that market sentiment cause price changes and vice versa were approved. In most research cases it is not clear which is the cause and which is the effect.

Challenges

Even though the correlation between price and sentiment in the market can be seen but predict the price of stocks is the hard part. Many types of research have been conducted in this field and results are somewhat average. 

Though another significant challenge linked to the sentiment analysis approach to the stock market is the notion of opinion expressing words. Some words can be perceived as either positive or negative depending on the context they are used, e.g the word short could be used in latency and indicates a positive message.

 

 

How Artificial Intelligence will revolutionize wealth management

/in /by

by Michal Rozanski, CEO at Empirica

Most wealth managers are in deep denial about robo advice. They say they need human interaction in order to understand the nuances of financial lives of their customers. And their clients value the human touch. They’re wrong. Soon robo advice will be much more efficient than human advice ever was.

In this post, we will share the results of our analysis on the most important areas where the application of machine learning will have the greatest impact in taking wealth management to the next level.

What Artificial Intelligence is and why you should care

 “Computers can only do what they are programmed to do.” Let us explain this is huge misconception, which was only valid because of limited processing power and memory capacity of computers. Most advanced programs which mimic specialized intelligences, known as expert systems, were indeed programmed around a set of rules based on the knowledge of specialists within the problem’s domain. There was no real intelligence there, only programmed rules. But there is another way to program computers, which makes them work more similarly to the functions of the human brain. It is based on showing the program examples of how certain problems can be solved and what results are expected. This way computers equipped with enough processing power, memory and storage are able to recognize objects in photographs, drive autonomous cars, recognize speech, or analyse any form of information which exhibits patterns.

 

We are entering the age where humans are outperformed by machines in activities related with reasoning based on the analysis of large amounts of information. Because of that finance and wealth management will be profoundly changed during the years to come.

 

Real advice – combining plans with execution

A great area for improvement in finance management is the combination of long term wealth building with the current financial situation of the customer as reflected by his bank account. For robo-advisors, an integration with bank API opens the door to an ocean of data which, after analysis, can dramatically improve the accuracy of advice provided to the customer.

By applying a machine learning capabilities to a customer’s monthly income and expenses data, wealth managers will gain a unique opportunity to combine two perspectives – the long term financial goals of their customers and their current spending patterns. Additionally, there is the potential of tax, mortgage, loans or credit card costs optimization, as well as using information on spending history to predict future expenditures.

By integrating data from social media, wealth management systems could detect major changes in one’s life situation, job, location, marital status or remuneration. This would allow for automated real time adjustments in investment strategies of on the finest level, which human advisors are simply unable to deliver.

New powerful tools in the wealth manager’s arsenal

Hedge funds that are basing their strategies on AI have provided better results over the last five years than the average (source Eurekahedge, more on hedge fund software). What is interesting is that the gap between AI and other strategies has been growing wider over the last two years, as advancements in machine learning accelerated.

The main applications of machine learning techniques in wealth management, can be categorized following cases:

  •       Making predictions on real-time information from sources such as market data, financial reports, news in different languages, and social media
  •       Analysis of historical financial data of companies to predict the company’s cash flow and important financial indicators based on the past performance of similar companies
  •       Analysis of management’s public statements and activity on social networks in order   to track the integrity of their past words, actions and results
  •       Help in accurate portfolio diversification by looking for uncorrelated instruments which match requirements of the risk profile (see portfolio management software)
  •       Generation of investment strategies parametrized by goals such as expected risk profiles, asset categories, and timespan, resulting in sets of predictive models which may be applied in order to fulfill the assumptions

 To give an example of machine learning accuracy, the algorithms for sentiment analysis and document classification are already on acceptable levels, well above 90%.

Automated execution

When it comes to the execution of the actual orders behind portfolio allocation and rebalancing strategies, many robo advisors are automating these processes passing generated orders to brokerage systems through algorithmic trading systems. The next step would be autonomous execution algorithms, that take under consideration the changing market situation and learn from incoming data, allowing for increased investment efficiency and reduced costs.

Machine learning can be applied to quantitative strategies like trend following, pattern recognition, mean reversion, and momentum, as well as the prediction and optimization of statistical arbitrage, and pairs trading. Additionally, there is a possibility to apply machine learning techniques in, already quite sophisticated, execution algorithms (aka trading bots) that help execute large orders by dividing them to thousands of smaller transactions without influencing the market while adjusting their aggressiveness to the market situation.

What’s interesting is that algorithms could also be trained to make use of rare events, like market crashes and properly react in milliseconds, already knowing the patterns of panic behaviour and shortages of liquidity provision.

Explaining the markets

In wealth management systems, if portfolio valuations are provided to the customers in real time, then so should explanations of the market situation. Every time the customer logs in to the robo-advisor, she should see all required portfolio information with a summary of market information relevant to the content of her portfolio. This process includes the selection of proper articles or reports concerning companies from the investor portfolio, classification and summarization of negative or positive news, and delivering a brief overview.

Additionally, machine learning algorithms can be used to discover which articles are read by customers and present only those type of articles that were previously opened and read by the customer.

The result will be not only the increase in customer understanding but also, by providing engaging content to investors, the increase in their engagement and commitment to portfolio strategy and wealth management services.

 

Talking with robots

The ability to deliver precise explanations of the market situation in combination with conversational interfaces aided by voice recognition technology will enable robo-advisors to provide financial advice in a natural, conversational way.

Voice recognition is still under development, but it could be the final obstacle on they way to redesigning human-computer interaction. On the other hand, thanks to deep learning, chatbot technology and question answering systems are getting more reliable than ever. KAI, the chatbot platform of Kasisto, who has been trained in millions of investment and trade interactions, already handles 95 % of all customer queries for India’s digibank.

Decreasing customer churn with behavioral analysis

The ability to track all customer actions, analyzing them, finding common patterns in huge amounts of data, making predictions, and offering unique insights for fund managers delivers a powerful business tool not previously available to wealth managers. What if nervousness caused by portfolio results or market situation could be observed in user behaviour within the system?  This information, combined with the results of investments and patterns of behaviour of other investors, can give a wealth manager the possibility to predict customer churn and react in advance.

When speaking with wealth management executives that are using our robo-advisory solutions, they indicate behavioural analysis as one of the most important advancements to their current processes. Customers leave not only when investment results are bad, but also when they are good if there is a fear that the results may not be repeated in the future. Therefore, the timely delivery of advice and explanations of market changes and the current portfolio situation are crucial.

The same model we used to solve the behavioral analysis problem has been proven to predict credit frauds in 93.07% of cases.

Summary

Other areas of applying machine learning in the processes supporting wealth management services could be:  

  •       Security based on fraud detection which actively learns to recognize new threats
  •       Improving sales processes with recommendations of financial products chosen by similar customers
  •       Psychological profiling of customers to better understand their reactions in different investment situations      
  •       Analysis and navigation of tax nuances   
  •       Real estate valuation and advice

Implementing these AI functions in wealth management systems will be an important step towards the differentiation of the wealth managers on the market. Today’s wealth managers’ tool set will look completely different in five years. Choosing an open and innovative robo-advisory system that tackles these future challenges is crucial. Equally important will be wealth managers’ incorporation of data analytic processes and the use of this data to help their customers.

Artificial intelligence is poised to transform the wealth management industry. This intelligence will be built on modern wealth management software that combine data from different sources, process it, and transform it into relevant financial advice. The shift from data gathering systems to predictive ones that help wealth managers to understand the data, has already started. And wealth management is all about understanding the markets and the customers.

 

 

The WEALTHTECH Book: The FinTech Handbook for Investors, Entrepreneurs and Finance Visionaries

/in , , /by

CEO at Empirica S.A. was a Co-Author of The WealthTech Book published in March 2018 by Wiley. He wrote a brilliant article on Robo-Advisors, which was placed in Chapter 67 – link to the release below ↓ https://lnkd.in/dUmfPt4

 

 

Empirica presented ‘ways of implementing Robo-Advisors’ at Fintech Trends

/in , , /by

On behalf of Empirica, Hanif Nezhad talked about the different ways Robo Advisors could be implemented and the lessons we have learnt from firms which have digitalized their wealth management operations

He used our Robo-Advisory platform to showcase the different ways possible to apply Robo-Advisors. Although the list of choices firms may have before going Robo is long, he used the major options available, such as model portfolio vs discretionary portfolio management, different ways to perform rebalancing and Empirica’s recommendation engine.

At the end of his presentation he showed how at Empirica we can improve advisor-customer relationship using our built-in house machine learning algorithm for Robo-Advisors.

The event was followed with a panel session where the three speakers discussed variety of topic related to Robo-Advisors such as differences among Robo-Advisors, the Robo-Advisor business in Europe and Poland. The panelists also talked about ETFs and why Robo-Advisors are a big fan of them. After the panel audiences could approach the speakers and ask their questions. The event was concluded with networking session.

Empirica was a proud sponsor of the event.

Empirica is a Wrocław-based company that supports many local IT initiatives. Empirica is offering solutions such as an Quant Trading Software implemented by major institutional investors in Poland, market making software, portfolio management system framework, crypto trading bots and trading software development for companies from capital and cryptocurrency markets.