News about Empirica, algorithmic trading and software development.

Machine Learning in Finance

/in , , , , , /by

It’s known that almost all industries are influenced or about to be influenced by the appliance of Artificial Intelligence. Perhaps operational efficiency is what makes Artificial Intelligence so attractive for business owners across different sectors. Operational efficiency could lead to the reduction of costs, increased performance, speed up some processes or increase the quality of services.

In this article, we would like to cover the appliance of Machine Learning across the financial industry by presenting interesting use cases and examples to structure this content. 

Artificial Intelligence is assisting financial institutions to drive new efficiencies and deliver new kinds of value. Autonomous Research predicts that Artificial Intelligence will represent $1 trillion in projected cost savings for the banking and financial services industry. By 2030, traditional financial institutions will save 22% of costs.

Let’s get something straight here and let’s define Machine Learning vs Artificial Intelligence. These two terms are always used side by side of each other, but they are different. With different, we mean that Machine Learning is a subset of Artificial Intelligence. Artificial Intelligence refers to create intelligent machines. Machine Learning refers to a system that can learn from experience. In this article, we may mention both but with the given simplistic definition you already know what we are referring to.

Machine Learning layers

Source

 

 

Challenges of applying Artificial Intelligence in finance

Before jumping into the use cases that we have gathered for this article, let’s take a look at the most common challenges of applying Artificial Intelligence in firms within the financial industry. 

As previously mentioned costs and budgeting required to automate some of the processes in finance could be one of the main important challenges financial firms face. Additionally, regulatory requirements sometimes could be also a burden. They are complex frameworks and the required research phase could be time-consuming and hiring regulatory consultants could be costly.

Perhaps another challenge these firms face is the lack of structured or sufficient data to process and train their data and test if models are efficient enough. Adding to that lack of in-house skills and knowledge as well as missing the development environment (lab) that data scientists can join and apply approaches of AI. 

Another challenge could also come from market maturity and retail readiness to use Artificial Intelligence-powered tools.  

Case 1. Fraud detection 

One of the very important appliances of Machine Learning in finance is fraud detection. With the advent of instant payment and global transfer services, the volume of payments and transfers has dramatically increased. So is a notable amount of transfers that aren’t with good intentions including money laundering. The estimated amount of money laundered globally in one year is 2-5% of global GDP, or $800 billion to $2trillion.  

An advantage that Machine Learning has brought to fraud detection is the amount of data that can be processed by machines with minimum or zero human intervention. The appliance comes with more accuracy in the detection of fraudulent activities. 

For instance, the credit card fraud detection problem includes modeling past credit card transactions with the knowledge of the ones that turned out to be a fraud. This model is then used to identify whether a new transaction is fraudulent or not. The aim here is to detect fraudulent transactions while minimizing incorrect fraud classifications. Anomaly detection is a commonly used model for the credit card fraud detection problem, this is a technique used to identify unusual patterns that do not conform to expected behavior, called outliers. Anomaly detection or pattern recognition algorithms start with creating processes that find the hidden correlation between each user behavior and classify the likelihood of fraudulent activity.  

Discovering hidden and indirect correlations can be named as advantages that Machine Learning based algorithms bring compare to basic Rule-based fraud detection algorithms. Machine Learning-based algorithms also reduce the number of verification measures since the intelligent algorithms fit with the behavior analytics of users. A more automated approach to detection of fraud is also another asset for Fraud Detection algorithms applied by Machine Learning since they require less manual work to enumerate all possible detection rules. 

Case 2. Know Your Customer (KYC)

Improving the KYC process is one of the operational efficiency of artificial intelligence and machine learning algorithms that have been brought to the financial and banking industry. The appliance of Machine Learning on the KYC process is mostly implemented by traditional banks and neobanks. The main reason is the continuous evolvement of requirements from regulators. The due diligence required on customer registration that is required by regulators in banking is broad and complex. Machine Learning due diligence modules can be utilized to create robust automation and improve the process of KYC for institutions that are aiming to have an efficient retail onboarding. This will decrease the human intervention needed during the onboarding process and increases the accuracy as well as reducing the costs. 

Again neobanks are streamlining the KYC process with enhanced user interface and user experience. Simplifying and automating the KYC process can reduce the cost of onboarding and customer application process by 40% (source Thomson Reuters). 

One of Machine learning techniques used in the KYC process is the Facial Similarity check, which is to verify that the face in the picture is the same with that on the submitted document provided e.g. Identity Card. The customer will only be verified and pass the KYC process if the results of both Document and Facial Similarity checks are ‘clear’. If the result of any check is not ‘clear’, the customer has to submit all the photos again.

Case 3. Algorithmic Trading

The algorithmic trading with a technological infrastructure brought many advantages to the trading world e.g. the ability to trade in under a millisecond with the best prices available or the ability to simultaneously monitor and trade across multiple exchanges, and all with reducing the human error from trading. Algorithmic trading constitutes 50-70% of the equity market trades and 60% of futures trades in developed markets. 

Many hedge funds started to utilize Artificial Intelligence within the algorithmic trading world. It’s understandable that most of them do not disclose the details and mechanism of their approaches in applying Artificial Intelligence in their trading algorithms, but it’s understood that they use methods of Machine Learning and Deep Learning. There is also a wide appliance of sentiment analysis on the market in which the result can be used in trading. The main objective of applying sentiment algorithms is to obtain knowledge about the psychology of the market. 

Machine Learning is assisting the trading industry in order to leverage the market with fundamental and alternative data in order to research alpha factors. Supervised, unsupervised and reinforcement learning models are being utilized to enhance the processing of algorithmic trading strategies. Methods can be applied to optimize portfolio risk calculations and further improve the performance of the portfolios. 

Deep Learning models also have been widely applied in trading. Deep learning models with multiple layers have shown as a promising architecture that can be more suitable for predicting financial time series data. In a tested practice, the algorithm trains 5-layer Deep Learning Network on high-frequency data of Apple’s stock price, and their trading strategy based on the Deep Learning produces 81% successful trade and a 66% of directional accuracy.

Case 4. Chatbots and customer support

Reducing customer churn is perhaps one of the main criteria of financial institutions and banks. Generally, customers and especially millennials ones for neobanks do care about the customer service and support they receive. Chatbots and instant messaging apps could potentially increase communication quality between business and its customers. According to research by Juniper by the year 2023, the use of chatbots can reduce the operational costs for banking, retail, and healthcare business sector by $11 billion. 

The advantages that chatbots can bring to the industry are definitely increasing customer satisfaction and customer engagement rates. The speed of action and processing of many inquiries and threads at the same time could also be mentioned as a big advantage of chatbots and messaging systems. 

There are four types that Chatbots could be classified. Goal-based Chatbots, are designed for a particular task and set up to have a short conversation in order to complete a task given to them by a user. Perhaps this is the most common Chatbot. Goal-based Chatbots are deployed on websites to help visitors to answer their questions during their visit. 

The second type is knowledge-based Chatbots, these Chatbots use the underlying data sources or the amount of data they are trained on. Such data sources could be open-domain or cloud domain data. Usually, Knowledge-based Chatbots answer questions by providing the data and source of that data. 

The third category of Chatbots is Serviced-based. Such Chatbots are classified based on facilities provided to the customer. It could be personal or commercial information. Users of such Chatbots could place an order of a commercial good via the Chatbot. 

The fourth category is Response-generated Chatbots. Response Generated-based chatbots are developed based on what action they perform in response generation. The response models take input and output in natural language text. The dialogue manager is responsible for combining response models together. To generate a response, the dialogue manager follows three steps.

Types-of-chatbots

Source

Case 5. Automated Wealth Management with Robo Advisors

Wealth management is an industry and operation costs are a large burden for some of the firms. As wealth being transferred from one generation to a more tech-savvy one and considering the millennials are in their prime earning and spending years, the presence of automated and entirely digital investment advice tools can be expected. It’s expected that by the year 2022 the Robo Advisors revenue might reach 25 billion that is up from $1.7 billion from 2017, considering these tools are relatively cheaper to how the investment advice is being delivered traditionally, from 3% to 5% of assets managed to digital ones with 0.25% to 0.75%. 

There are practiced used cases where the processes of asset allocation modeling, portfolio construction, and optimization, as well as a portfolio recommendation systems, were bundled with Machine Learning techniques in order to enhance the current approaches. 

One example is the portfolio recommendation system that was designed to be implemented on top of a Robo Advisor and be utilized with the mean-variance optimization method was implemented using weighted linear regression. The model shows that adding a portfolio layer on top of the stock regression results is increasing the success rate (profit accuracy) up to 86.69% when success is calculated by the profitability of the recommendations. Moreover, it helps to reduce the risk by distributing the budget over a set of stocks and tries to minimize the reflection of the regression errors to the profit.

Robo Advisors come with many notable advantages, such a complete, online and real-time reporting dashboard to customers which can be checked on the go with mobile apps and dashboards. 

As previously mentioned they reduce the costs of operations for firms providing investment advice service and the fees that are clients charged. Robo Advisors are fully digital and they have online onboarding for clients which leads to expansion of client base for firms.

When can you apply AI (is your firm ready?)

There are few aspects to which we could measure the readiness of a firm to utilize Artificial Intelligence and Machine learning into their processes. A solid technological infrastructure is the most important element. An infrastructure that is put together to manage the whole lifecycle of data, from getting to cleaning to processing and feeding algorithms. The availability of the data can not be stressed more. 

The regulatory compliance as mentioned in some of the cases above e.g. in KYC processes is a crucial process to be taken care of before applying Artificial Intelligence into processes. Audit trails, transparency, result supervision, and reporting mechanisms are some of the high-level requirements from financial authorities. 

Talents as resources from data engineers to data scientists specialized and familiar with financial processes is another important criteria before kick-starting with Artificial Intelligence projects. Their ability to understand the sector and ways to improve it should be taken into account in their hiring process. Eventually, they need to start training the existing data with an accuracy level as a requirement for the models used. 

How to start your machine learning project?

  • Start with a question 

Before anything starts you need to start with the question, what is it that we want to improve with our Machine Learning algorithms? This should specify and clarify the objective of the project. 

  • Understand your data

Not every question can be answered with any data. You need to have the right data for the right question. This is practiced by receiving, cleaning and processing data. Running exploratory analysis on your data and making sense of some of the summaries obtained could be the initial stage to which you will know that if your data has the potential to answer your questions. 

  • Modeling

Once you found clues in your data associated with your question it’s time to try to write algorithms to find patterns that leads to successful or unsuccessful journeys. Usually, data scientists do this by fitting the most suitable Machine Learning models into the data, find correlative and statistically significant patterns and try to test the accuracy. 

  • Evaluation

In the modeling section, we talked about training your data and once we have found the best models that suit the question, the answer and the data and now its time to evaluate, in other words, test your models. Data scientists will keep on testing the models with new data to see if their models do not only work for one dataset. 

  • Deployment

Once the fitting algorithms are certain that works, it’s time to deploy. Generally, this means deploying a code representation of the model into an operating system to score or categorize new unseen data as it arises and to create a mechanism for the use of that new information in the solution of the original business problem.  Importantly, the code representation must also include all the data preparation steps leading up to modeling so that the model will treat new raw data in the same manner as during model development.

 

Empirica among the winners of the Deloitte Technology Fast 50

/in /by
Read more

Deribit – A derivative Cryptocurrency exchange

/in /by

Introducing Deribit Cryptocurrency Exchange:

Deribit is a cryptocurrency derivatives exchange allowing traders to trade Bitcoin and Ethereum Options, Perpetual and futures. Deribit keeps its customers’ deposits in their cold storage with most of the funds stored in vaults with multiple banks safes. Though at this point in time, Deribit only accepts Bitcoin and no fiat currency as funds to deposit. 

 

Deribit Futures:

Futures on Deribit receive a cash settlement instead of physical Bitcoin, that means the buyer of the futures do not buy the actual Bitcoin and the seller won’t sell the actual BTC and what will be transferred is the transfer of profits and losses at the agreed settlement price of the contract on the expiration price. 

The minimum tick size to trade futures is 0.50 USD and all daily settlements happen at the coordinated universal time (UTC) 8 am. The expiration time is also at UTC 8 am at the end of each month. The size for the contracts at Deribit future exchange is 10 USD with initial margin starting at 1.0% with a linear increment of 0.5% per 100 BTC. The delivery price is the Time Weighted Average of Deribit BTC index measured in the half an hour before the UTC 8 am (7:30 am).

 

Deribit Perpetual

At Deribit exchange, Perpetual is a derivative very similar to Futures but with no fixed maturity and no exercise limit. The perpetual derivatives are to keep their price close to their underlying cryptocurrency price, which at Deribit exchange is referred to as “Deribit BTC Index”.

The Deribit Perpetual contract is 1 USD per Index Point with a contract size of 10 USD. The minimum tick size 0.50 USD and settlements are done at UTC 8 am. The contract size for trading Prepetuals in Deribit is 10 USD. As mentioned earlier there is no delivery/expiration when trading Prepetuals on Deribit. 

 

Deribit Options:

Options on Deribit are traded with what so-called the “European style”. This indicates that options cannot be exercised before expiration, but can only be exercised at expiration. Which this happens automatically on Deribit.  Options on Deribit are priced in Bitcoin and Ethereum and also viewable on USD.

BTC option deribit

Deribit Index:

There are 8 exchanges and the highest and lowest prices are taken out, and the remaining 6 are each at 16.67% accountable for creating an index in Deribit. 

Deribit Crypto Index

 

Market Making on Deribit:

Deribit does not include an in-house trading desk, therefore all active market makers are the third party Market Makers. The liquidity is provided by these parties and Deribit sees these services as a crucial point to their business. Based on the volume, designated market makers receive tailored agreement on fees. 

Market sentiment analysis

/in /by

What is market sentiment analysis? 

Also known as opinion mining or emotion artificial intelligence, or textual sentiment analysis aims to process and extract the subjective content of descriptive and written work. This objective is achieved through the analysis of the source’s opinions, or their evaluation towards a topic or a product as well as sentiments, attitudes, and emotions. 

In trading, investment banks and hedge funds are trying to take advantage of sentiments of the market to help them make better predictions about the financial market. Some of the very accomplished firms including DE Shaw, Two Sigma and Renaissance Technologies have been reported to use sentiment signals. In some of these approaches, sentiment signals are blended in with other data such as transactional data (prices, historical returns or dividends). Sentiment analysis can be performed as stand-alone research prior to traditional trading or it could be equipped into algorithmic trading. 

Techniques such as Natural Language Processing and Text Mining are employed in order to perform analysis and the extraction process. 

Types of sentiment

Based on the typology proposed by Scherer, there are two types of sentiment while performing sentiment analysis. One type is the “attitude”, which is a narrow definition of sentiment, whether the opinion is “positive” or “negative”.

The second type of sentiment within sentiment analysis is the “emotion”. This indicates the eight “basic emotions”, which are in four opposing pairs, joy-sadness, anger-fear, trust-disgust and anticipation-surprise. 

Steps to perform sentiment analysis on market data

Step one: data collection

In order to perform sentiment analysis on market instruments e.g. a stock, there first should be the source to which the data stream flows for sentiment algorithms to start processing. Such data sources are best to be provided through APIs to reach automation in the collection process. One common and widely used source of data for stock market sentiment analysis is Twitter. Through the search query API provided by Twitter, the thrid-party system can query data via the REST API with given conditions such as location, language and time and etc. 

Step two: Classifier engine

Once required corpus data created for the sentiment analysis, there should be a step to extract features from the data. Different domains should design classifiers differently since words could mean differently across different industries. Having said that, most of the sentimental words (basically adjectives) are self-explanatory. Some of the methods used to create word classifiers and feature extraction engines are Naive base, Support Vector Machine (SVM) or K-clustering methods. 

Step three: predictive modeling

Once the classifier engine is in place, the data scientists will start the process of examining the best predictive models for the data. Which makes this step into two phases, the training, and testing (predicting). during the training phase, the data scientists will train different models and fine-tune them in order to use it for the second phase which is testing which will result in obtaining an accuracy level. This process iterates until there is an acceptable accuracy level determined by domain experts and data scientists. Results of such tests should also be checked if whether they are statistically reproducible or not, basically if the model achieves a similar accuracy level on different datasets throughout time.

Are price and sentiment correlated?

There are numerous researches conducted over this question, nevertheless, the general intuition in the market is that; there is a correlation between sentiment and price in the market. In experimental work with consideration to asses whether the financial market reacts to relevant news events, the sentiment analysis was performed using both a standard model and an enhanced temporal model. Within the temporal test, they associate the sentiments with the corresponding temporal orientations by classifying each sentence with one of the four temporal categories (past/present/future/unknown) and calculate the sentiment strength accordingly.

 

Correlation-market-sentiment-price

The result of this practice concluded that the casualty test experiment, two competing hypotheses that market sentiment cause price changes and vice versa were approved. In most research cases it is not clear which is the cause and which is the effect.

Challenges

Even though the correlation between price and sentiment in the market can be seen but predict the price of stocks is the hard part. Many types of research have been conducted in this field and results are somewhat average. 

Though another significant challenge linked to the sentiment analysis approach to the stock market is the notion of opinion expressing words. Some words can be perceived as either positive or negative depending on the context they are used, e.g the word short could be used in latency and indicates a positive message.

How Artificial Intelligence will revolutionize wealth management

/in , , , , , /by

by Michal Rozanski, CEO at Empirica

Most wealth managers are in deep denial about robo advice. They say they need human interaction in order to understand the nuances of financial lives of their customers. And their clients value the human touch. They’re wrong. Soon robo advice will be much more efficient than human advice ever was.

In this post, we will share the results of our analysis on the most important areas where the application of machine learning will have the greatest impact in taking wealth management to the next level.

What Artificial Intelligence is and why you should care

 “Computers can only do what they are programmed to do.” Let us explain this is huge misconception, which was only valid because of limited processing power and memory capacity of computers. Most advanced programs which mimic specialized intelligences, known as expert systems, were indeed programmed around a set of rules based on the knowledge of specialists within the problem’s domain. There was no real intelligence there, only programmed rules. But there is another way to program computers, which makes them work more similarly to the functions of the human brain. It is based on showing the program examples of how certain problems can be solved and what results are expected. This way computers equipped with enough processing power, memory and storage are able to recognize objects in photographs, drive autonomous cars, recognize speech, or analyse any form of information which exhibits patterns.

 

We are entering the age where humans are outperformed by machines in activities related with reasoning based on the analysis of large amounts of information. Because of that finance and wealth management will be profoundly changed during the years to come.

 

Real advice – combining plans with execution

A great area for improvement in finance management is the combination of long term wealth building with the current financial situation of the customer as reflected by his bank account. For robo-advisors, an integration with bank API opens the door to an ocean of data which, after analysis, can dramatically improve the accuracy of advice provided to the customer.

By applying a machine learning capabilities to a customer’s monthly income and expenses data, wealth managers will gain a unique opportunity to combine two perspectives – the long term financial goals of their customers and their current spending patterns. Additionally, there is the potential of tax, mortgage, loans or credit card costs optimization, as well as using information on spending history to predict future expenditures.

By integrating data from social media, wealth management systems could detect major changes in one’s life situation, job, location, marital status or remuneration. This would allow for automated real time adjustments in investment strategies of on the finest level, which human advisors are simply unable to deliver.

New powerful tools in the wealth manager’s arsenal

Hedge funds that are basing their strategies on AI have provided better results over the last five years than the average (source Eurekahedge, more on hedge fund software). What is interesting is that the gap between AI and other strategies has been growing wider over the last two years, as advancements in machine learning accelerated.

The main applications of machine learning techniques in wealth management, can be categorized following cases:

  •       Making predictions on real-time information from sources such as market data, financial reports, news in different languages, and social media
  •       Analysis of historical financial data of companies to predict the company’s cash flow and important financial indicators based on the past performance of similar companies
  •       Analysis of management’s public statements and activity on social networks in order   to track the integrity of their past words, actions and results
  •       Help in accurate portfolio diversification by looking for uncorrelated instruments which match requirements of the risk profile (see portfolio management software)
  •       Generation of investment strategies parametrized by goals such as expected risk profiles, asset categories, and timespan, resulting in sets of predictive models which may be applied in order to fulfill the assumptions

 To give an example of machine learning accuracy, the algorithms for sentiment analysis and document classification are already on acceptable levels, well above 90%.

Automated execution

When it comes to the execution of the actual orders behind portfolio allocation and rebalancing strategies, many robo advisors are automating these processes passing generated orders to brokerage systems through algorithmic trading systems. The next step would be autonomous execution algorithms, that take under consideration the changing market situation and learn from incoming data, allowing for increased investment efficiency and reduced costs.

Machine learning can be applied to quantitative strategies like trend following, pattern recognition, mean reversion, and momentum, as well as the prediction and optimization of statistical arbitrage, and pairs trading. Additionally, there is a possibility to apply machine learning techniques in, already quite sophisticated, execution algorithms (aka trading bots) that help execute large orders by dividing them to thousands of smaller transactions without influencing the market while adjusting their aggressiveness to the market situation.

What’s interesting is that algorithms could also be trained to make use of rare events, like market crashes and properly react in milliseconds, already knowing the patterns of panic behaviour and shortages of liquidity provision.

Explaining the markets

In wealth management systems, if portfolio valuations are provided to the customers in real time, then so should explanations of the market situation. Every time the customer logs in to the robo-advisor, she should see all required portfolio information with a summary of market information relevant to the content of her portfolio. This process includes the selection of proper articles or reports concerning companies from the investor portfolio, classification and summarization of negative or positive news, and delivering a brief overview.

Additionally, machine learning algorithms can be used to discover which articles are read by customers and present only those type of articles that were previously opened and read by the customer.

The result will be not only the increase in customer understanding but also, by providing engaging content to investors, the increase in their engagement and commitment to portfolio strategy and wealth management services.

 

Talking with robots

The ability to deliver precise explanations of the market situation in combination with conversational interfaces aided by voice recognition technology will enable robo-advisors to provide financial advice in a natural, conversational way.

Voice recognition is still under development, but it could be the final obstacle on they way to redesigning human-computer interaction. On the other hand, thanks to deep learning, chatbot technology and question answering systems are getting more reliable than ever. KAI, the chatbot platform of Kasisto, who has been trained in millions of investment and trade interactions, already handles 95 % of all customer queries for India’s digibank.

Decreasing customer churn with behavioral analysis

The ability to track all customer actions, analyzing them, finding common patterns in huge amounts of data, making predictions, and offering unique insights for fund managers delivers a powerful business tool not previously available to wealth managers. What if nervousness caused by portfolio results or market situation could be observed in user behaviour within the system?  This information, combined with the results of investments and patterns of behaviour of other investors, can give a wealth manager the possibility to predict customer churn and react in advance.

When speaking with wealth management executives that are using our robo-advisory solutions, they indicate behavioural analysis as one of the most important advancements to their current processes. Customers leave not only when investment results are bad, but also when they are good if there is a fear that the results may not be repeated in the future. Therefore, the timely delivery of advice and explanations of market changes and the current portfolio situation are crucial.

The same model we used to solve the behavioral analysis problem has been proven to predict credit frauds in 93.07% of cases.

Summary

Other areas of applying machine learning in the processes supporting wealth management services could be:  

  •       Security based on fraud detection which actively learns to recognize new threats
  •       Improving sales processes with recommendations of financial products chosen by similar customers
  •       Psychological profiling of customers to better understand their reactions in different investment situations      
  •       Analysis and navigation of tax nuances   
  •       Real estate valuation and advice

Implementing these AI functions in wealth management systems will be an important step towards the differentiation of the wealth managers on the market. Today’s wealth managers’ tool set will look completely different in five years. Choosing an open and innovative robo-advisory system that tackles these future challenges is crucial. Equally important will be wealth managers’ incorporation of data analytic processes and the use of this data to help their customers.

Artificial intelligence is poised to transform the wealth management industry. This intelligence will be built on modern wealth management software that combine data from different sources, process it, and transform it into relevant financial advice. The shift from data gathering systems to predictive ones that help wealth managers to understand the data, has already started. And wealth management is all about understanding the markets and the customers.

 

 

Now Crypto. Lessons learned from over 10 years of developing trading software

/in , , , , /by

By Michal Rozanski, CEO at Empirica.

Reading news about crypto we regularly see the big money inflow to new companies with a lot of potentially breakthrough ideas. But aside from the hype from the business side, there are sophisticated technical projects going on underneath.

And for new cryptocurrency and blockchain ideas to be successful, these projects have to end with the delivery of great software systems that scale and last. Because we have been building these kinds of systems for the financial markets for over 10 years we want to share a bit of our experience.

Read more on how Empirica delivers its trading software development services

Read more

“Software is eating the world”. I believe these words by Marc Andreessen. And now the time has come for financial markets, as technology is transforming every corner of the financial sector. Algorithmic trading, which is our speciality, is a great example. Other examples include lending, payments, personal finance, crowdfunding, consumer banking and retail investments. Every part of the finance industry is experiencing rapid changes triggered by companies that propose new services with heavy use of software.

If crypto relies on software, and there is so much money flowing into crypto projects, what should be looked for when making a trading software project for cryptocurrency markets? Our trading software development projects for the capital and crypto markets as well as building our own algorithmic trading platform has taught us a lot. Now we want to share our lessons learned from these projects.

 

  1. The process – be agile.

Agile methodology is the essence of how software projects should be made. Short iterations. Frequent deliveries. Fast and constant feedback from users. Having a working product from early iterations, gives you the best understanding of where you are now, and where you should go.

It doesn’t matter if you outsource the team or build everything in-house; if your team is local or remote. Agile methodologies like Scrum or Kanban will help you build better software, lower the overall risk of the project and will help you show the business value sooner.

 

  1. The team – hire the best.

A few words about productivity in software industry. The citation is from my favourite article by Robert Smallshire ‘Predictive Models of Development Teams and the Systems They Build’ : ‘… we know that on a small 10 000 line code base, the least productive developer will produce about 2000 lines of debugged and working code in a year, the most productive developer will produce about 29 000 lines of code in a year, and the typical (or average) developer will produce about 3200 lines of code in a year. Notice that the distribution is highly skewed toward the low productivity end, and the multiple between the typical and most productive developers corresponds to the fabled 10x programmer.’.

I don’t care what people say about lines of code as a metric of productivity. That’s only used here for illustration.

The skills of the people may not be that important when you are building relatively simple portals with some basic backend functionality. Or mobile apps. But if your business relies on sophisticated software for financial transactions processing, then the technical skills of those who build it make all the difference.

And this is the answer to the unasked question why we in Empirica are hiring only best developers.

We the tech founders tend to forget how important it is to have not only best developers but also the best specialists in the area which we want to market our product. If you are building an algo trading platform, software for market makers or trading bots, you need quants. If you are building banking omnichannel system, you need bankers. Besides, especially in B2B world, you need someone who will speak to your customers in their language. Otherwise, your sales will suck.

And finally, unless you hire a subcontractor experienced in your industry, your developers will not understand the nuances of your area of finance.

 

  1. The product – outsource or build in-house?

If you are seriously considering building a new team in-house, please read the points about performance and quality, and ask yourself the question – ‘Can I hire people who are able to build systems on required performance and stability levels?’. And these auxiliary questions – can you hire developers who really understand multithreading? Are you able to really check their abilities, hire them, and keep them with you? If yes, then you have a chance. If not, better go outsource.

And when deciding on outsourcing – do not outsource just to any IT company hoping they will take care. Find a company that makes systems similar to what you intend to build. Similar not only from a technical side but also from a business side.

Can outsourcing be made remotely without an unnecessary threat to the project? It depends on a few variables, but yes. Firstly, the skills mentioned above are crucial; not the place where people sleep. Secondly, there are many tools to help you make remote work as smooth as local work. Slack, trello, github, daily standups on Skype. Use it. Thirdly, find a team with proven experience in remote agile projects. And finally – the product owner will be the most important position for you to cover internally.

And one remark about a hidden cost of in-house development, inseparably related to the IT industry – staff turnover costs. Depending on the source of research, turnover rates for software developers are estimated at 25% to even 38%. That means that when constructing your in-house team, every fourth or even every third developer will not be with you in a year from now. Finding a good developer – takes months. Teaching a new developer and getting up to speed – another few months. When deciding on outsourcing, you are also outsourcing the cost and stress of staff turnover.

 

  1. System’s performance.

For many crypto projects, especially those related with trading,  system’s performance is crucial. Not for all, but when it is important, it is really important. If you are building a lending portal, performance isn’t as crucial. Your customers are happy if they get a loan in a few days or weeks, so it doesn’t matter if their application is processed in 2 seconds or in 2 minutes. If you are building an algo trading operations or bitcoin payments processing service, you measure time in milliseconds at best, but maybe even in nanoseconds. And then systems performance becomes a key input to the product map.

95% of developers don’t know how to program with performance in mind, because 95% of software projects don’t require these skills. Skills of thinking where bytes of memory go, when they will be cleaned up, which structure is more efficient for this kind of operation on this type of object. Or the nightmare of IT students – multithreading. I can count on my hands as to how many people I know who truly understand this topic.

 

  1. Stability, quality and level of service.

Trading understood as an exchange of value is all about the trust. And software in crypto usually processes financial transactions in someway.

Technology may change. Access channels may change. You may not have the word ‘bank’ in your company name, but you must have its level of service. No one in the world would allow someone to play with their money. Allowing the risk of technical failure may put you out of business. You don’t want to spare on technology. In the crypto sapce there is no room for error.

You don’t achieve quality by putting 3 testers behind each developer. You achieve quality with processes of product development. And that’s what the next point is about.

 

  1. The DevOps

The core idea behind DevOps is that the team is responsible for all the processes behind the development and continuous integration of the product. And it’s clear that agile processes and good development practices need frequent integrations. Non-functional requirements (stability and performance) need a lot of testing. All of this is an extra burden, requiring frequent builds and a lot of deployments on development and test machines. On top of that there are many functional requirements that need to be fulfilled and once built, kept tested and running.

On many larger projects the team is split into developers, testers, release managers and system administrators working in separate rooms. From a process perspective this is an unnecessary overhead. The good news is that this is more the bank’s way of doing business, rarely the fintech way. This separation of roles creates an artificial border when functionalities are complete from the developers’ point of view and when they are really done – tested, integrated, released, stable, ready for production. By putting all responsibilities in the hands of the project team you can achieve similar reliability and availability, with a faster time to the market. The team also communicates better and can focus its energy on the core business, rather than administration and firefighting.

There is a lot of savings in time and cost in automation. And there are a lot of things that can be automated. Our DevOps processes have matured with our product, and now they are our most precious assets.

 

  1. The technology.

The range of technologies applied for crypto software projects can be as wide as for any other industry. What technology makes best fit for the project depends, well, on the project. Some projects are really simple such as mobile or web application without complicated backend logic behind the system. So here technology will not be a challenge. Generally speaking, crypto projects can be some of the most challenging projects in the world. Here technologies applied can be the difference between success and failure. Need to process 10K transaction per second with a mean latency under 1/10th ms. You will need a proven technology, probably need to resign from standard application servers, and write a lot of stuff from scratch, to control the latency on every level of critical path.

Mobile, web, desktop? This is more of a business decision than technical. Some say the desktop is dead. Not in trading. If you sit whole day in front of the computer and you need to refer to more than one monitor, forget the mobile or web. As for your iPhone? This can be used as an additional channel, when you go to a lunch, to briefly check if the situation is under control.

 

  1. The Culture.

After all these points up till now, you have a talented team, working as a well-oiled mechanism with agile processes, who know what to do and how to do it. Now you need to keep the spirits high through the next months or years of the project.

And it takes more than a cool office, table tennis, Xbox consoles or Friday parties to build the right culture. Culture is about shared values. Culture is about a common story. With our fintech products or services we are often going against big institutions. We are often trying to disrupt the way their business used to work. We are small and want to change the world, going to war with the big and the powerful. Doesn’t it look to you like another variation of David and Goliath story? Don’t smile, this is one of the most effective stories. It unifies people and makes them go in the same direction with the strong feeling of purpose, a mission. This is something many startups in other non fintech branches can’t offer. If you are building the 10th online grocery store in your city, what can you tell your people about the mission?

Read more on how Empirica delivers its crypto software development services

Read more

 

Final words

Crypto software projects are usually technologically challenging. But that is just a risk that needs to be properly addressed with the right people and processes or with the right outsourcing partner. You shouldn’t outsource the responsibility of taking care of your customers or finding the right market fit for your product. But technology is something you can usually outsource and even expect significant added value after finding the right technology partner.

At Empirica we have taken part in many challenging crypto projects, so learn our lessons, learn from others, learn your own and share it. This cycle of learning, doing and sharing will help the crypto community build great systems that change the rules of the game in the financial world!

 

 

Independent initiatives that analyze crypto exchanges liquidity and quality

/in , , , , , , , /by

Volume is flawed metric of crypto exchanges liquidity. Because of wash trading practices of many crypto exchanges as well as token issuers, using trading volume as a basis of comparison is misleading. Many exchanges have problems attracting professional market makers and are trying to make shortcuts on the way to attract retail investors. Moreover attracting professional investors requires investments in crypto exchanges system development with stable and performant APIs so they could connect their algorithmic trading systems.]

There are more and more independent initiatives that are taking a closer look at what constitutes a high quality crypto exchange. Three major ones are Blockchain Transparency Institute, CryptoCompare Benchmark and Cointelligence Report. I also take a quick look at the Bitwise report for SEC from March 2019.

 

Read more about our tool for measuring crypto exchange quality – Liquidity Analytics Dashboard

Read more

 

Blockchain Transparency Institute

BTI concentrates on analyzing crypto exchanges data feeds to spot wash trading mechanisms and provide the real volume metric which is cleaned out of suspicious activities.

BTI identified 17 of the CoinMarketCap Top 25 crypto exchanges to be over 99% wash traded. This one number alone shows the magnitude of the problem, as well as how volume is a false measure.

According to BTI Report crypto exchanges which are faking their volumes use a variety of different tactics to try and swindle investors. These tactics include buying twitter followers and likes, filling up fake order books, mirror wash trading the largest exchanges with real volume, and trying to disguise their wash trading using various bot settings to not affect price. On many of these exchanges trading high volumes closing the spread would make the volume plummet as the trading bots had no room to wash trade with themselves. Welcome to the wild wild west of no regulation and surveillance.

BTI finds that “all crypto exchanges combined are currently reporting around $50 Billion in daily volume on CMC. After removing all the wash traded volume via our algorithms the accurate number is around $4-5 Billion. About 88-92% of daily trading volume is fabricated depending on the day. Bitcoin’s daily trading volume is about 92% fabricated, which is in line with the space as a whole when comparing our findings to top data sites reporting wash traded volumes.” 

And further “On our list of the top 40 largest exchanges with actual volume, Bitcoin’s volume is about 65% fabricated. Almost all of this fabricated volume comes from OKEx, Bibox, HitBTC, and Huobi. Of the top 25 tokens by market cap, Tron and Ethereum Classic are the highest wash traded tokens on our list at 85% fake volume each and coming in at #24 and #25 of the most wash traded tokens.”

Top 10 cryptocurrency exchanges according to real (not wash traded) volume by BTI

  1. Binance 
  2. Kucoin
  3. Liquid
  4. Huobi
  5. Coinbase
  6. OKEx
  7. Bitfinex
  8. Upbit
  9. Kraken
  10. Bitstamp

CryptoCompare

CryptoCompare’s Exchange Ranking methodology utilises a combination of 34 qualitative and quantitative metrics to assign a grade to over 100 active crypto exchanges. Metrics were categorised into several buckets ensuring that no one metric overly influences the overall exchange ranking. Each crypto exchange grade is derived from a broad due diligence check using qualitative data, followed by a market quality analysis that uses a combination of order book and transactional data.

Due diligence check comprises of 6 main categories that attempt to qualitatively rate each exchange on the basis of:

  • Geography
  • Legal and regulatory metrics
  • Calibre of investment
  • Team and company quality
  • Quality of data provision
  • Trade surveillance

Although at Empirica we believe in numbers, I like the qualitative approach, as it’s also possible to prove a correlation of metric like number of employees and business size of the exchange, therefore proving this way it’s quality. 

Another important factor is Market Quality. Crypto compare measures the market quality of each exchange using a combination of 5 metrics (derived from trade and order book data) that aim to measure the:

  • Cost to trade, 
  • Liquidity, 
  • Market stability, 
  • Behaviour towards sentiment
  • “Natural” trading behaviour

Exchanges were rated based on a combination of 9 of the most liquid BTC and ETH markets.

It’s worth taking a closer look how CryptoCompare report approaches Spread and Liquidity metrics:

“Generally, those exchanges which offer incentives to provide liquidity through either low or negative maker fees will achieve the tightest spreads. Due to the spread being calculated using the best bid and offer, it is misleading to use it as a sole gauge of liquidity and therefore as the market cost to trade; it must be used in conjunction with a depth

measurement to find the likely transaction price for any given size of transaction.”

 

Good point. And liquidity:

“Market depth is the total volume of orders in the order book. It provides an idea of how much it is possible to trade on crypto exchange, and how much the price is likely to move if large amounts are traded. An exchange with greater average depth is likely to be more stable (i.e flash crashes are much less likely) and allows trading of greater amounts at better prices.

We consider the depth up to 1% either side of the mid price. 

Depth = E(depthUp+depthDown)/2

Where depthUp is the total volume that would be required to move the price by 1% upwards from the mid price, and

depthDown is the total volume that would be required to move the price by 1% downwards from the mid price.”

 

Top 10 crypto exchanges according CryptoCompare quality benchmark:

  1. Coinbase 
  2. Poloniex 
  3. Bitstamp 
  4. bitFlyer 
  5. Liquid
  6.  itBit 
  7. Kraken 
  8. Binance 
  9. Gemini 
  10. Bithumb 

 

Cointelligence Rating System

Cointelligence is the most qualitative rating of crypto exchanges from the above. The methodology of the team was to manaully open accounts on all analyzed crypto exchanges and check from the user perspective the core aspects of beeing an exchange customer. The aspects cover:

Usability – covers KYC process, the quality of exchange website, extent of features and how easy it is to get a human answer from support staff. 

Performance – functionalities and historical robustness of exchange matching engine, fees height, trading instruments like futures contracts and margin trading.

Team – analysis of the available information about management team behind the crypto exchange, especially business and technical experience of C-level staff, including person responsible for exchange’s security

Risk – information on past hacks, insurance status, account security layers but also regulatory status of cryptocurrency exchange. Based on the geographical location of the exchange headquarters and registration any potential run-ins with the local law or any sign of authorities involvement.

 

This way Contelligence analyzed 85 crypto exchanges, but only 15 is rated with good quality mark, lead by Liquid and Gemini. 

Top 10 cryptocurrency exchanges by Cointelligence by qualitative criteria 

  1. Liquid (Quoine)
  2. Gemini
  3. Binance
  4. Bitstamp
  5. Gibraltar Blockchain Exchange
  6. OKEx
  7. Bittrex
  8. itBit
  9. Kraken
  10. ABCC

Bitwise report for SEC

Bitwise analysis is based on detecting wash trading patterns in public marked data published by crypto exchanges. Out of 81 exchanges they have analyzed in March 2019 only 10 were identified as be free of wash trading practices. These exchanges are:

  1. Binance
  2. Bitfinex
  3. Kraken
  4. Bitstamp
  5. Coinbase
  6. bitFlyer
  7. Gemini
  8. itBit
  9. Bitrex
  10. Poloniex

Bitwise identified that only 4,5% (about $275M daily) of officially reported volume (eg by the public sources like coinmarketcap) is the actual volume. The rest is wash traded.

The Bitcoin market is more orderly and efficient than is commonly understood. The 10 exchanges trade as a uniform, highly connected market. They form a singular price. Average deviations from the aggregate price for the ten exchanges is well within the expected arbitrage band when you account for exchange-level fees (~30 basis points), volatility and hedging costs. Arbitrage is operating well. Sustained deviations (defined as deviations >1% that last more than 100 seconds) appear as single white lines on the graph below. The graph demonstrates that the ten exchanges trade at a single unified price.

So although the message about the amount of wash traded volume is alarming, the report shows that the real crypto market is quite concentrated, ordered, efficient and well performing. The rest is just noise.

 

 

Read more about our tool for measuring crypto exchange quality – Liquidity Analytics Dashboard

Read more

 

 

Blockchain meetup sponsored by Empirica, Wroclaw

/in , , , , , , /by

Monday June 19th a beautiful sunny day in IT-friendly Wroclaw, tech start-ups and cryptocurrency enthusiast gather together at IT corner Tech meetup, sponsored by Empirica.

The event was planned to focus on key areas of current trends in Blockchain and Ethereum.

The event began with Mr Wojciech Rokosz, Ardeo CEO presentation. The session was dedicated to introduction to the economics of token. Explaining the new changes and updates we are and we will face in our economy with this huge entrance of virtual currencies.

The event later carried on with Mr Marek Kotewicz on introduction to Blockchain, Bitcoin and Ethereum. The session was summarizing the differences between Bitcoin and Ethereum.

The third and last part of the event was conducted with Mr Tomek Drwga, Blockchain meetup organizer,  diving deeper into smart contracts and programming ( introduction to Solidity) for Ethereum.

The event ended with open discussion between the audience and speakers, and visitors were served with beverages.

Empirica is a Wrocław-based company that supports many local IT initiatives. Empirica is offering solutions such as an Algo Trading Software implemented by major institutional investors in Poland, market makers software, wealth management system framework, cryptocurrency trading bots and trading software development services for companies from capital and cryptocurrency markets.

 

The WEALTHTECH Book: The FinTech Handbook for Investors, Entrepreneurs and Finance Visionaries

/in /by

CEO at Empirica S.A. was a Co-Author of The WealthTech Book published in March 2018 by Wiley. He wrote a brilliant article on Robo-Advisors, which was placed in Chapter 67 – link to the release below ↓ https://lnkd.in/dUmfPt4

New York Intensive Business Journey | Consensus 2019

/in /by

During our last visit to New York, we held multiple business meetings with our partners and potential clients, which led to kick-start some new, exciting algo trading projects.

We had been spreading the word about our flagship products – Algorithmic Trading Engine, Liquidity Engine and the newborn baby – Liquidity Analytics Dashboard for crypto markets. Making use of every spare hour, we participated in different industry events connected with crypto trading and blockchain.

You might have met Empirica’s Vice-President and Co-Funder Piotr Stawiński on conferences and meetups such as NYC Crypto Mondays, various Blockchain Week events or Consensus 2019.

Empirica is a Poland based company that supports many local IT initiatives. Empirica is offering solutions such as an Algo Trading Software implemented by major institutional investors in Poland, market makers software, wealth management system framework, best cryptocurrency trading bots and trading software development services for companies from capital and cryptocurrency markets.