Articles related to software development

Posts

FinTech. Lessons learned from over 5 years of financial technology software projects.

By Michal Rozanski, CEO at Empirica.

 

Reading news about fintech we regularly see the big money inflow to new companies with a lot of potentially breakthrough ideas. But aside from the hype from the business side, there are sophisticated technical projects going on underneath. And for new fintech ideas to be successful, these projects have to end with the delivery of great software systems that scale and last. Because we have been building these kind of systems for the fintech area for over 5 years we want to share a bit of our experience.

 

fintech empirica

 

“Software is eating the world”. I believe these words by Marc Andreessen. And now the time has come for finance, as technology is transforming every corner of the financial sector. Algorithmic trading, which is our speciality, is a great example. Other examples include lending, payments, personal finance, crowdfunding, consumer banking and retail investments. Every part of the finance industry is experiencing rapid changes triggered by companies that propose new services with heavy use of software.
The best evidence that something is happening somewhere is to see where the money goes. Investments in fintech companies globally grew to $12 billion last year, which is a three times increase comparing to 2013, and five times during the last five years, according to the research reports by CBInsights.

If fintech relies on software, and there is so much money flowing into fintech projects, what should be looked for when making a fintech software project? Our outsourcing software projects for the fintech industry as well as building our own algorithmic trading platform has taught us a lot. Now we want to share our lessons learned from these projects.

 

1. The process – be agile.

Agile methodology is the essence of how software projects should be made. Short iterations. Frequent deliveries. Fast and constant feedback from users. Having a working product from early iterations, gives you the best understanding of where you are now, and where you should go.
It doesn’t matter if you outsource the team or build everything in-house; if your team is local or remote. Agile methodologies like Scrum or Kanban will help you build better software, lower the overall risk of the project and will help you show the business value sooner.

 

2. The team – hire the best.

A few words about productivity in software industry. The citation is from my favourite article by Robert Smallshire ‘Predictive Models of Development Teams and the Systems They Build’ : ‘… we know that on a small 10 000 line code base, the least productive developer will produce about 2000 lines of debugged and working code in a year, the most productive developer will produce about 29 000 lines of code in a year, and the typical (or average) developer will produce about 3200 lines of code in a year. Notice that the distribution is highly skewed toward the low productivity end, and the multiple between the typical and most productive developers corresponds to the fabled 10x programmer.’.
I don’t care what people say about lines of code as a metric of productivity. That’s only used here for illustration.
The skills of the people may not be that important when you are building relatively simple portals with some basic backend functionality. Or mobile apps. But if your business relies on sophisticated software for financial transactions processing, then the technical skills of those who build it make all the difference.

And this is the answer to the unasked question why we in Empirica are hiring only best developers.

We the tech founders tend to forget how important it is to have not only best developers but also the best specialists in the area which we want to market our product. If you are building an algo trading platform, you need quants. If you are building banking omnichannel system, you need bankers. Besides, especially in B2B world, you need someone who will speak to your customers in their language. Otherwise, your sales will suck.
And finally, unless you hire a subcontractor experienced in your industry, your developers will not understand the nuances of your area of finance.

 

3. The product – outsource or build in-house?

If you are seriously considering building a new team in-house, please read the points about performance and quality, and ask yourself the question – ‘Can I hire people who are able to build systems on required performance and stability levels?’. And these auxiliary questions – can you hire developers who really understand multithreading? Are you able to really check their abilities, hire them, and keep them with you? If yes, then you have a chance. If not, better go outsource.
And when deciding on outsourcing – do not outsource just to any IT company hoping they will take care. Find a company that makes systems similar to what you intend to build. Similar not only from a technical side but also from a business side.
Can outsourcing be made remotely without an unnecessary threat to the project? It depends on a few variables, but yes. Firstly, the skills mentioned above are crucial; not the place where people sleep. Secondly, there are many tools to help you make remote work as smooth as local work. Slack, trello, github, daily standups on Skype. Use it. Thirdly, find a team with proven experience in remote agile projects. And finally – the product owner will be the most important position for you to cover internally.

And one remark about a hidden cost of in-house development, inseparably related to the IT industry – staff turnover costs. Depending on the source of research, turnover rates for software developers are estimated at 25% to even 38%. That means that when constructing your in-house team, every fourth or even every third developer will not be with you in a year from now. Finding a good developer – takes months. Teaching a new developer and getting up to speed – another few months. When deciding on outsourcing, you are also outsourcing the cost and stress of staff turnover.

 

4. System’s performance.

For many fintech areas system’s performance is crucial. Not for all, but when it is important, it is really important. If you are building a lending portal, performance isn’t as crucial. Your customers are happy if they get a loan in a few days or weeks, so it doesn’t matter if their application is processed in 2 seconds or in 2 minutes. If you are building an algo trading operations or payments processing service, you measure time in milliseconds at best, but maybe even in nanoseconds. And then systems performance becomes a key input to the product map.
95% of developers don’t know how to program with performance in mind, because 95% of software projects don’t require these skills. Skills of thinking where bytes of memory go, when they will be cleaned up, which structure is more efficient for this kind of operation on this type of object. Or the nightmare of IT students – multithreading. I can count on my hands as to how many people I know who truly understand this topic.

 

5. Stability, quality and level of service.

Finance is all about the trust. And software in fintech usually processes financial transactions in someway.
Technology may change. Access channels may change. You may not have the word ‘bank’ in your company name, but you must have its level of service. No one in the world would allow someone to play with their money. Allowing the risk of technical failure may put you out of business. You don’t want to spare on technology. In the fintech sector there is no room for error.

You don’t achieve quality by putting 3 testers behind each developer. You achieve quality with processes of product development. And that’s what the next point is about.

 

6. The Dev Ops

The core idea behind DevOps is that the team is responsible for all the processes behind the development and continuous integration of the product. And it’s clear that agile processes and good development practices need frequent integrations. Non-functional requirements (stability and performance) need a lot of testing. All of this is an extra burden, requiring frequent builds and a lot of deployments on development and test machines. On top of that there are many functional requirements that need to be fulfilled and once built, kept tested and running.

On many larger projects the team is split into developers, testers, release managers and system administrators working in separate rooms. From a process perspective this is an unnecessary overhead. The good news is that this is more the bank’s way of doing business, rarely the fintech way. This separation of roles creates an artificial border when functionalities are complete from the developers’ point of view and when they are really done – tested, integrated, released, stable, ready for production. By putting all responsibilities in the hands of the project team you can achieve similar reliability and availability, with a faster time to the market. The team also communicates better and can focus its energy on the core business, rather than administration and firefighting.

There is a lot of savings in time and cost in automation. And there are a lot of things that can be automated. Our DevOps processes have matured with our product, and now they are our most precious assets.

 

7. The technology.

The range of technologies applied for fintech software projects can be as wide as for any other industry. What technology makes best fit for the project depends, well, on the project. Some projects are really simple such as mobile or web application without complicated backend logic behind the system. So here technology will not be a challenge. Generally speaking, fintech projects can be some of the most challenging projects in the world. Here technologies applied can be the difference between success and failure. Need to process 10K transaction per second with a mean latency under 1/10th ms. You will need a proven technology, probably need to resign from standard application servers, and write a lot of stuff from scratch, to control the latency on every level of critical path.

Mobile, web, desktop? This is more of a business decision than technical. Some say the desktop is dead. Not in trading. If you sit whole day in front of the computer and you need to refer to more than one monitor, forget the mobile or web. As for your iPhone? This can be used as an additional channel, when you go to a lunch, to briefly check if the situation is under control.

 

8. The Culture.

After all these points up till now, you have a talented team, working as a well-oiled mechanism with agile processes, who know what to do and how to do it. Now you need to keep the spirits high through the next months or years of the project.
And it takes more than a cool office, table tennis, play station or Friday parties to build the right culture. Culture is about shared values. Culture is about a common story. With our fintech products or services we are often going against big institutions. We are often trying to disrupt the way their business used to work. We are small and want to change the world, going to war with the big and the powerful. Doesn’t it look to you like another variation of David and Goliath story? Don’t smile, this is one of the most effective stories. It unifies people and makes them go in the same direction with the strong feeling of purpose, a mission. This is something many startups in other non fintech branches can’t offer. If you are building the 10th online grocery store in your city, what can you tell your people about the mission?

 

Final words

Fintech software projects are usually technologically challenging. But that is just a risk that needs to be properly addressed with the right people and processes or with the right outsourcing partner. You shouldn’t outsource the responsibility of taking care of your customers or finding the right market fit for your product. But technology is something you can usually outsource and even expect significant added value after finding the right technology partner.
At Empirica we have taken part in many challenging fintech projects, so learn our lessons, learn from others, learn your own and share it. This cycle of learning, doing and sharing will help the fintech community build great systems that change the rules of the game in the financial world!

 

 

Free version of Algorithmic Trading Platform for retail investors

We have just released beta of Empirica – Algorithmic Trading Paltform for retail investors! It’s lifetime free for development, testing and optimizing of trading algorithms.

Our development team (exactly this team who implemented the entire system) also provides full support in algorithms development as well as connectivity to brokers. If you need help just contact us.

Among many features what is unique is our exchange simulation where you can influence market conditions under which you test your algorithms. No others software offers such a realistic level of simulation.

In paid versions we offer the execution of algorithms in robust server side architecture.

We strive for your feedback!

Best regards,

Michal Rozanski
Founder and CEO at Empirica
twitter: @MichalRoza
http://empirica.io


Empirica Trading Platform – http://empirica.io

Our platform implemented by large brokerage house!

Empirica has successfuly finished the implementation of its Algorithmic Trading Platform in one of the largest brokerage houses in Poland.

Brokerage house will use our software to:

  • aid its internal trading operations, like market making of derivatives on Warsaw Stock Exchange
  • offer functionalities of our platform to its institutional clients, which will be able to build, test and execute their own algorithmic trading strategies

Implementation included connecting of our software system directly to the system of Warsaw Stock Exchange (Universal Trading Platform delivered by NYSE Technologies), as well as the integration with transaction systems of brokerage house. Additionally we have fulfilled and successfuly passed tests regarding the highest security, stability and performance requirements.

This implementation is an important milestone for our system. The usage by team of market makers is a proof that our system is capable of performing high-throughput and low latency operations on level required by most sophisticated traders on the capital marketets.

 

Next release of our algorithmic platform. Version 1.3.4 – has code name “The Firebolt”.

Next implementations of our Algorithmic Trading Platform by customers don’t stop us from developing the platform itself. Working agile requires us to keep the pace in short and frequent iterations, which in case of product means frequent releases, keeping the whole product line stable.

A few iterations that we planned in our 1.3.4 release, code named by our developers ‘The Firebolt’, will include among others:

– even faster real-time replication of all server-side components in master-slave mode (for deployment in larger institutions)

– extended client side backtesting capabilities

– sophisticated charting of backtesting results and statistics

– multiscreen mode of client side application

– additional web-based server-side module for administration & management

 

For those curious about the release name and unfamiliar with Harry Potter, Firebolt is:

“The state-of-the-art racing broom. The Firebolt has unsurpassable balance and pinpoint precision. Aerodynamic perfection.”
—Harry Potter: Quidditch World Cup

“The Firebolt has an acceleration of 150 miles an hour in ten seconds and incorporates an unbreakable Braking Charm. Price upon request.”
—Harry reads about the features of the Firebolt.

Speed, precision, balance, perfection. These are the words that describe our software, therefore choosing the code name was kind of obvious :).

 

The Firebolt broom

Warsaw Stock Exchange certifies our Trading Platform

 

Empirica’s Algorithmic Trading Platform has successfully passed the XDP protocol communication certification, issued by the Warsaw Stock Exchange.

From now on Empirica is officially listed as the ISV (Independent Software Vendor) for the Warsaw Stock Exchange.

WSE uses Universal Trading Platform delivered by NYSE Technologies. The same system is used by many other European and world stock exchanges. Fulfilment of technical criteria of Warsaw Stock Exchange makes certification for those markets only a formality for our platform.

Empirica founds IT Corner association to support local entrepreneurship!

Empirica, along with several other software companies based in Wroclaw, has founded the IT Corner association.

IT Corner aims at supporting the development of local IT enterprises, tightening the cooperation among small and medium size high-tech sector companies and developing project and product synergies between organization members.

IT Corner will pursue it goals by:

  • organization of IT events and conferences for larger audience
  • regular technological meet-ups targeted at employees of IT Corner companies
  • cooperate on larger IT projects with member companies
  • know-how and best-practices sharing among management of member companies
  • common presence on job fairs, IT events in Poland and abroad

First common events are already planned and will be officially announced soon!

it_corner_100x100-01

The list of founding members encloses over 10 software companies employing altogether over 200 people. Till end of the year IT Corner aims to double its size and establish its position as biggest and most active IT association in Wroclaw.

More on: IT Corner site

TWAP Strategy

Time-Weighted Average Price (TWAP) is another trading algorithm based on weighted average price and in compare to Volume-Weighted Average Price its calculations are even simplier. Also it’s one of the first execution algorithms and unlike most algorithms nowadays it’s passive execution algorithm that waits for proper market price to come, doesn’t chase it.

 

Calculations

 

As TWAP doesn’t bother about volume it’s extremely simple to obtain it. All it takes is to get Typical Price for every period bar using equation below and then calculate average of Typical Prices.

 

Typical Price = (Close+High+Low+Open)/4

 

Let’s just take a look at example results calculated on 1-minute interval intraday Morgan Stanley’s stock.

 

Time Close High Low Open Typical Price TWAP
09:30:00 38.90 38.96 38.90 38.96 38.93 38.930
09:31:00 38.94 38.97 38.86 38.92 38.92 38.926
09:32:00 38.91 38.96 38.91 38.94 38.93 38.928
09:33:00 38.89 38.94 38.88 38.92 38.91 38.922
09:34:00 38.90 38.94 38.90 38.90 38.91 38.920
09:35:00 38.97 38.97 38.90 38.90 38.93 38.922
09:36:00 38.92 38.96 38.92 38.96 38.94 38.925
09:37:00 38.90 38.93 38.86 38.93 38.91 38.922
09:38:00 38.90 38.92 38.89 38.89 38.90 38.920
09:39:00 38.92 38.92 38.88 38.91 38.91 38.918
09:40:00 38.90 38.92 38.88 38.91 38.90 38.917
09:41:00 38.84 38.89 38.82 38.89 38.86 38.912
09:42:00 38.87 38.87 38.84 38.84 38.86 38.908
09:43:00 38.85 38.89 38.84 38.89 38.87 38.905
09:44:00 38.81 38.85 38.80 38.85 38.83 38.900
09:45:00 38.69 38.80 38.67 38.80 38.74 38.890

 

Strategy

 

The most common use of TWAP is for distributing big orders throughout the trading day. For example let’s say you want to buy 100,000 shares of Morgan Stanley. Putting one such a big order would vastly impact the market and the price most likely would start to raise. To prevent that, investor can define time period in TWAP Strategy over which they want to buy shares. It will slice evenly big order into smaller ones and execute them over defined period.

 

TWAP could be used as alternative to VWAP, but because of itssimplicity we have to remember about some pitfalls. Even if we slice big orders, we do it evenly, thus there is a possibility to hit on low liquidity period when our splitted order will impact the market hard. That’s why it’s recommended to use TWAP over short periods or on stocks that are believed to not have any volume profile to follow.

 

Be random

 

There is also another threat coming directly from dividing big order evenly, namely, other traders or predatory algorithms. Obviously trading in such a predictable way can lead to situation where other traders or algorithms would look through our strategy and start to “game” us.

 

Barry Johnson in his book suggests adding some randomness to the strategy as a solution to the issue. He says that “We can use the linear nature of the target completion profile to adopt a more flexible trading approach. At any given time, we can determine the target quantity the order should have achieve just by looking up the corresponding value on the completion rate chart.”

 

In practice it means that when we have run 4-hour TWAP we don’t slice the order into evenly parts, but otherwise we focus on percentage completion. So for instance we would want to have 25% of the strategy completed by first hour, 50% by second and 75% by third. That gives a more freedom into size of orders, so we can be more random with it and hence less predictable for other traders on the market.

 

TWAP vs VWAP

 

As both indicators use same mechanism, i.e. weighted average price, it’s common to compare them. Despite that VWAP’s nature is more complex and includes volume in its calculations, on  instruments with low turnover TWAP and VWAP values can be close. On the other hand when a session starts to be more volatile both indicators will diverge.

 

 

On a table below there are TWAP and VWAP calculated for whole trading day. As we can see at the beginning of the trading day the difference is less than a cent, but on close the difference raised up to 2 cents. It happened because during the day there were some small volume trades for lower price that didn’t affected VWAP, but did TWAP.

 

Time Close High Low Open TWAP VWAP
09:44:00 38.81 38.85 38.80 38.85 38.900 38.904
09:45:00 38.69 38.80 38.67 38.80 38.890 38.887
15:57:00 38.70 38.70 38.68 38.69 38.666 38.686
15:58:00 38.71 38.72 38.68 38.70 38.666 38.686

 

Summary

 

TWAP Strategy is another great tool for executing big orders without impacting the market too hard. Like everything it has its own pros and cons and it’s up to us to select if TWAP will be the best strategy to use for our case or maybe we should consider using VWAP or other strategy.

 

References

  1. H. Kent Baker, Greg Filbeck. “Portfolio Theory of Management” (2013) , pp.421
  2. Barry Johnson “Algorithmic & Trading DMA – An introduction to direct access trading strategies” (2010), pp. 123-126

 

 

Basics of High Frequency Trading

Nanex’s High Frequency Trading Model (Sped Up)

Nanex released a video showing the results of half a second of worldwide high frequency trading with Johnson and Johnson stock. I simply sped up the footage to get a better feel of what it looked like. Blow Your Mind.

High frequency trading in action

CNN’s Maggie Lake gets a rare look inside the super-fast trading industry.

High Frequency Trading Explained (HFT)
Dave Fry, founder and publisher of ETF Digest, and Steve Hammer, founder of HFT Alert, discuss high frequency trading operations, fundamentals, the difference between algorithmic trading and high frequency trading, fluttering, latency and the role high frequency trading had in the May stock market flash crash in 2010.
 
TEDxNewWallStreet – Sean Gourley – High frequency trading and the new algorithmic ecosystem

Dr. Sean Gourley is the founder and CTO of Quid. He is a Physicist by training and has studied the mathematical patterns of war and terrorism. He is building tools to augment human intelligence.

Watch high-speed trading in action

Citadel Group, a high-frequency trading firm located in Chicago, trades more stocks each day than the floor of the NYSE.

Wild High Frequency Trading Algo Destroys eMini Futures

One of the scariest high frequency trading algos ran in the electronic S&P 500 futures (eMini) contract on January 14, 2008 starting at 2:01:11Eastern. During its 7 second reign, there were over 7,000 trades (52,000 contracts), and the price eventually oscillated within milliseconds, the equivalent of about 400 points in the Dow Jones Industrial Average!


HFT trading ideally must have the lowest possible info latency (time delays) and the maximum potential automation level. So participants prefer to trade in markets with high levels of integration and automation capacities in their trading platforms. These include NYSE NASDAQ, Direct Edge and BATS.
HFT is controlled by proprietary trading firms and spans across multiple securities, including equities, derivatives, index funds and ETFs, currencies and fixed income instruments.For HFT, participants want the following infrastructure in place:
– High speed computers, which need costly and regular hardware upgrades;
– Co-location.
– Real time data feeds, which must avert even the delay which could affect profits; and of a microsecond
– Computer algorithms, which are the heart of HFT and AT.

Benefits of HFT
– HFT is beneficial to traders, but does it help the total marketplace? Some market that is overall gains that HFT assistants cite contain:
– Bid-ask spreads have reduced due to HFT trading, making markets more efficient. Empiric evidence contains that after Canadian authorities in April 2012 imposed fees that deterred HFT, studies indicated that “the bid-ask spread rose by 9%,” possibly due to diminishing HFT trades. And thus facilitates the effects of market fragmentation.

– HFT assists in the price discovery and price formation process, as it is centered on a high number of orders (see related: How The Retail Investor Profits From High Frequency Trading.)

Basics of Machine Learning in Algorithmic Trading

Algorithmic Game Theory and Practice, Michael Kearns, University of Pennsylvania,

Traditional financial markets have undergone rapid technological change due to increased automation and the introduction of new mechanisms. Such changes have brought with them challenging new problems in algorithmic trading, many of which invite a machine learning approach. I will briefly survey several algorithmic trading problems, focusing on their novel ML and strategic aspects, including limiting market impact, dealing with censored data, and incorporating risk considerations.

 

 

Machine learning for algorithmic trading w/ Bert Mouler

Harnessing the power of machine learning for money making algo strategies with Bert Mouler

Practical Tips For Algorithmic Trading (Using Machine Learning)
Evgeny Mozgunov from Caltech won an algorithmic trading competition hosted by Quantiacs. Jenia used machine learning tools to write his trading algorithm that now trades an initial $1M investment. He is talking about his approach and his main learnings. Jenia’s algorithm currently has a live Sharpe Ratio of 2.66.
 
Machine Learning and Pattern Recognition for Algorithmic Forex and Stock Trading
This is a whole course (20 videos) on machine learning and algorithmic trading. In this series, you will be taught how to apply machine learning and pattern recognition principles to the field of stocks and forex. This is especially useful for people interested in quantitative analysis and algo trading. Even if you are not, the series will still be of great use to anyone interested in learning about machine learning and automatic pattern recognition, through a hands-on tutorial series.
 

The following frontier of the technological arms race in finance is artificial intelligence. Improvements in AI research have triggered massive curiosity about the sector, where some consider a a trading, learning and thinking computer will make even today’s superfast, ultra-complicated investment algorithms appear archaic — and potentially leave human fund managers redundant. Could the next generation’s Buffett be a super-algo?

Some of the world’s largest cash managers are betting on it. AI investing may sound as, although fantastical sci-fi writer William Gibson said The future is already here, it’s simply not evenly dispersed.” Bridgewater, the world’s biggest hedge fund group, poached the head of IBM’s artificial intelligence unit Watson in 2012, and Two Sigma and last year BlackRock, another rapidly growing hedge fund that uses quantitative models, hired two former top Google engineers. Headhunters say computer scientists are now the hottest property in finance.

The quantitative investment world plays down the prospect of machines arguing that human genius still plays an important part, pointing out that the prospect of artificial intelligence that is complete is still distant, and supplanting human fund managers. But the confident swagger of the cash management nerds is unmistakable. There are quasi-AI trading strategies working their magic and the future belongs to them, they predict.

Artificial intelligence and video that is financePlay
The time will come that no human investment manager will manage to defeat the computer.” Or, as Agent Smith put it succinctly in The Matrix: “Never send a human to do a machine’s occupation.”

Yin Luo first learned to code after an used Apple II computer was brought back to China by his dad from a business trip to West Germany in 1985, when he was 11. But there were no games to purchase his home town in Heilongjiang province, in Yichun, so he made a crude variation of Tanks, where the player shoots down randomly created aeroplanes and taught himself to program.
It was arduous work. The computer’s lack of memory meant it crashed the program coding grew not too simple. He’d no floppy discs, so he learnt how to save the info on cassette tapes. “I just really needed something to play with,” Mr Luo recalls.

But the expertise paid off. Now, he is part of a growing tribe of brainiacs on Wall Street investigating the bleeding edge of computer science.

A network of 20 Linux servers is needed to run the hyper-sensible “ adaptive style turning” that was linear model, which is founded on a “machine learning” algorithm.

Machine learning is a branch of AI a diffuse term that is certainly frequently misused or misunderstood. While many people comprehend AI to mean sentient computers like the archvillains SkyNet in the Terminator films or HAL 9000 in 2001: A Space Odyssey, in practice everyday tools including Google’s language translation service, Netflix’s film recommendation engine or Apple’s Siri virtual assistant install basic forms of AI.

Quants have long used increasingly powerful computers to crunch numbers and uncover statistical signs of money-making opportunities, but machine learning goes a step further.

It can learn the difference between apples and bananas and sort out them, or perhaps instruct a computer how to play and quickly master a game like Super Mario from scratch. Machine learning can also be unleashed on “unstructured data”, such as for example jumbled amounts but also pictures and videos which can be typically not easy for a computer to comprehend.

More powerful computers mean that it are now able to be applied to financial markets, although the technique is old. “It’s a very bright area,” Mr Luo says. “Artificial intelligence is able to help you find designs an individual would never see. That may give you an enormous advantage.”

But that is not the only advantage of machine learning.

When marketplaces undergo what industry participants call a regime change that is “ ” and trusted strategies no more use, one of the classic challenges for quants is that their models can often prove worthless — or worse. Algorithmic trading strategies that print cash one day can blow the next up.

A machine learning algorithm adjusting to what works in markets that day, will autonomously develop and search for new patterns.
That means they can be used by asset managers as something to develop trade and strategies by itself, or perhaps to enhance their investment process, maybe by screening for patterns undetectable by people.

For Nick Granger, a fund manager at Man AHL, a quant hedge fund, that is the advantage that is critical. “You see it creating intuitive trading strategies in the bottom up, changing styles according to what works,” he says. “ We have been using machine learning for the past few years and have an interest in investing it in more.”

Nonetheless, machine learning has pitfalls. One of the largest challenges for quants is a phenomenon called “overfitting”, when an algorithm that is coded or exceedingly complicated finds false signals or specious correlations in the noise of data. For instance, a blog called “Spurious Correlations” notes that margarine consumption is linked to Nick Cage pictures to swimming pool drownings, and divorce rates in Maine.

When confronted by actual markets even if your model functions well in testing it can fail. Also, new data can be changed by the trading algorithm, says Osman Ali, a quant at Goldman Sachs’ asset management arm. “ you’re not affecting the weather, but if you deal marketplaces they are being affected by you If you crunch weather data.”

Nor can the most complex AI think as creatively as an individual, especially in a crisis. Brad Betts, a former Nasa computer scientist now working at BlackRock’s “ active equity” arm that is scientific, emphasizes the 2009 emergency plane landing on the Hudson river by Chesley Sullenberger of when machine is trumped by man as an example.

Truly, some quants remain sceptical that machine learning — AI or more broadly — is a holy grail for investment. Many see it just as a fresh, sophisticated gizmo to supplement their present toolkit, but others claim it truly is mainly a case of intelligent marketing rather than something genuinely ground-breaking.

They do’t constantly work, although “People are always desperate to find new ways to earn money in financial markets. He points out the human brain is uniquely adept at pattern recognition, “whether it really is love, a triangle or a face. Investment management is totally amenable to being addressed by computers designed to see patterns, but I’m not going to rush to use the latest hot algo to do so.”

 

Basics of algorithmic trading

Algo-trading provides these advantages:

 
– Trades executed at the best possible prices
– Immediate and accurate trade arrangement positioning (thereby high likelihood of performance at desired amounts)
– Trades timed right and forthwith, to avert significant cost changes
– Reduced transaction costs (see the execution shortfall example below)
– Coincident automated tests on multiple marketplace states
– Decreased risk of manual errors in placing the trades
– Backtest the algorithm, depending on available historical and real time data
– Reduced chance of errors by human traders based on variables that are mental and emotional
  

Algorithmic Trading Strategies

Any strategy for algorithmic trading requires an identified chance which will be rewarding when it comes to improved gains or cost reduction. The following are common trading strategies used in algo trading:
 
Trend Following Strategies
The most common algorithmic trading strategies follow fads in moving station breakouts, averages, price level moves and technical indicators that are related. These are most straightforward and the easiest strategies to execute through algorithmic trading because these strategies don’t involve making any predictions or price outlooks. Trades are commenced depending on the incidence of desired tendencies, which are easy and straightforward without getting into the complexity of predictive analysis to implement. The aforementioned example of 200 and 50 day moving average is a popular trend following strategy.
 
Arbitrage Opportunities
Purchasing a dual listed stock at a lower cost in one market and simultaneously selling it at an increased price in another marketplace offers the price differential as risk free gain or arbitrage. The same operation can be duplicated for stocks versus futures instruments, as price differentials do exists from time to time. Implementing an algorithm to identify such price differentials and placing the orders enables lucrative opportunities in efficient manner.
 
Index Fund Rebalancing
Index funds have defined periods of rebalancing to bring their holdings to level with their respective benchmark indices. This creates opportunities that are lucrative for algorithmic dealers, who capitalize on anticipated trades that offer 20-80 basis points gains depending upon how many stocks in the index fund, only prior to index fund rebalancing. Such trades are initiated via algorithmic trading systems for best costs and timely performance.
 
Mathematical Model Based Strategies
A lot of proven mathematical models, like the delta-neutral trading strategy, which enable trading on combination of its underlying security and alternatives, where trades are placed to cancel positive and negative deltas so that the portfolio delta is kept at zero.
 
Trading Range (Mean Reversion)
Mean reversion strategy is dependant on the idea the low and high costs of an asset are a temporary phenomenon that revert to their mean value periodically. Identifying and explaining a price range and implementing algorithm based on that allows trades to be put automatically when cost of advantage breaks in and out of its defined range.
 
Percent of Volume (POV)
Until the commerce order is fully filled, this algorithm continues sending partial orders, based on the defined contribution ratio and according to the volume traded in the markets.
 
Implementation Shortfall
The implementation shortfall strategy aims at minimizing the performance cost of an order by trading off the real time marketplace, thereby saving on the cost of the order and benefiting in the opportunity cost of delayed performance. The participation speed that is targeted will be increased by the strategy when the stock price moves positively and decrease it when the stock price moves adversely.
 
Beyond the Usual Trading Algorithms
There are a few special classes of algorithms that try to identify “happenings” on one other side. These “sniffing algorithms,” used, for instance, by a sell side market maker have the inbuilt intelligence to identify the existence of any algorithms on the buy side of a large order. Such detection through algorithms will help the market maker identify large order opportunities and enable him to gain by filling the orders at a cost that is higher. This is occasionally identified as high-tech front-running. (For more on high frequency trading and deceptive practices, see: If You Buy Stocks Online, You Are Involved in HFTs.)
 
Time Weighted Average Price (TWAP)
Time weighted average cost strategy breaks up a large order and releases determined smaller balls of the order to the marketplace using equally split time slots between a start and ending time. The intention is to carry out the order close to the average cost between the end and start times, thereby minimizing market impact.
 
Volume Weighted Average Price (VWAP)
Quantity weighted average cost strategy breaks up a large order and releases determined smaller balls of the order to the market using stock particular historic volume profiles. The aim would be to carry out the order close to the Volume Weighted Average Price (VWAP), therefore gaining on average cost.

Educational material for basics of algorithmic trading

 
TEDxNewWallStreet – Sean Gourley – High frequency trading and the new algorithmic ecosystem
 
See the video by Dr. Sean Gourley. He is the founder and CTO of Quid and physicist by training and has studied the mathematical patterns of war and terrorism. He is building tools to augment human intelligence.
 

 

Algorithmic Trading– Impact of Automated Trading Programs On Markets Documentary

 

Learn about the impact of automated trading systems on today’s markets. While this documentary focuses on stocks, the same factors are at work in the Forex markets. High frequency, algorithmic trading programs work quickly and create huge volatility. This excellent documentary Money & Speed is from VPRO which is required viewing for all traders.

 
 
Documentary: Money & Speed: Inside the Black Box
 
Money & Speed: Inside the Black Box is a thriller based on actual events that takes you to the heart of our automated world. Based on interviews with those directly involved and data visualizations up to the millisecond, it reconstructs the flash crash of May 6th 2010: the fastest and deepest U.S. stock market plunge ever. A rare opportunity to experience what is happening inside the black boxes of our rapidly evolving financial markets.
 

 

TEDxConcordia – Yan Ohayon – The Impact of Algorithmic Trading

 

Yan Ohayon demystifies and shares his experience with algorithmic trading and its impact on markets, our lives, and everything in between.

 

Quants: The Alchemists of Wall Street – A Documentary about algorythmic trading