Articles related to algo trading and software tools aiding automated investment operations.
|We have just released beta of Empirica – Algorithmic Trading Paltform for retail investors! It’s lifetime free for development, testing and optimizing of trading algorithms.
Our development team (exactly this team who implemented the entire system) also provides full support in algorithms development as well as connectivity to brokers. If you need help just contact us.
Among many features what is unique is our exchange simulation where you can influence market conditions under which you test your algorithms. No others software offers such a realistic level of simulation.
In paid versions we offer the execution of algorithms in robust server side architecture.
We strive for your feedback!
Empirica Trading Platform – http://empirica.io
Empirica has successfuly finished the implementation of its Algorithmic Trading Platform in one of the largest brokerage houses in Poland.
Brokerage house will use our software to:
- aid its internal trading operations, like market making of derivatives on Warsaw Stock Exchange
- offer functionalities of our platform to its institutional clients, which will be able to build, test and execute their own algorithmic trading strategies
Implementation included connecting of our software system directly to the system of Warsaw Stock Exchange (Universal Trading Platform delivered by NYSE Technologies), as well as the integration with transaction systems of brokerage house. Additionally we have fulfilled and successfuly passed tests regarding the highest security, stability and performance requirements.
This implementation is an important milestone for our system. The usage by team of market makers is a proof that our system is capable of performing high-throughput and low latency operations on level required by most sophisticated traders on the capital marketets.
Next implementations of our Algorithmic Trading Platform by customers don’t stop us from developing the platform itself. Working agile requires us to keep the pace in short and frequent iterations, which in case of product means frequent releases, keeping the whole product line stable.
A few iterations that we planned in our 1.3.4 release, code named by our developers ‘The Firebolt’, will include among others:
– even faster real-time replication of all server-side components in master-slave mode (for deployment in larger institutions)
– extended client side backtesting capabilities
– sophisticated charting of backtesting results and statistics
– multiscreen mode of client side application
– additional web-based server-side module for administration & management
For those curious about the release name and unfamiliar with Harry Potter, Firebolt is:
“The state-of-the-art racing broom. The Firebolt has unsurpassable balance and pinpoint precision. Aerodynamic perfection.”
—Harry Potter: Quidditch World Cup
“The Firebolt has an acceleration of 150 miles an hour in ten seconds and incorporates an unbreakable Braking Charm. Price upon request.”
—Harry reads about the features of the Firebolt.
Speed, precision, balance, perfection. These are the words that describe our software, therefore choosing the code name was kind of obvious :).
The Firebolt broom
High Frequency Trading, known also as HFT, is a technology of market strategies execution. HFT is defined by technically simple and time costless algorithms that run on appropriate software optimized for data structures, level of memory usage and processor use, as well as suitable hardware, co-location and ultra low-latency data feeds.
Although HFT exists on the market for over 20 years, it has became one of the hottest topic during past few years. It is caused by several factors, such as May 6, 2010, “Flash crash”, latest poor situation on the market and Michael Lewis book – “Flash Boys”. Let’s look where all that fuss comes from.
Among other things, the advantage over other market participants and ability to detect market inefficiencies is the reason why so many people critics HFT so much. Most common charges put on the table are:
- Front Running – HFT companies use early access to incoming quotes to buy shares before other investors and then turn around and sell him just bought shares with slightly bigger price.
- Quote Stuffing – Way of market manipulation by quick sending and withdrawing large number of orders. Because of speed of operations, it creates a false impression of the situation on the market that leads other participants to executing against phantom orders. Then there is nothing else to do, but to exploit favorable prices by HFT investors.
- Spoofing – Another method for market manipulation by placing orders and then cancelling them for price increase/decrease. It is based on placing big order on the market to bait other investors, and when the market starts to react, quickly cancel it. Then new price allows to gain some profit by HFT investor.
But that’s just a tip of the iceberg. It can be often heard that there is lack of proper HFT regulations, exist false belief that there are Dark Pools without any regulations where HFT companies can hide their activity, and there is still active argument if HFT brings liquidity to the market or just useless volume.
Bill Laswell once said “People are afraid of things they don’t understand. They don’t know how to relate. It threatens their security, their existence, their career, image.” That phrase perfectly fits to what is happening now on High Frequency Trading topic. When people would like to take a closer look on how exchanges work, probably, they would be less sceptic to High Frequency Trading.
Thus, on most, maybe even on all, exchanges exist two mechanism which can efficiently handle problem of quote stuffing and spoofing. First of them is limitation of number of messages per second that can be send from one client. For example on New York Stock Exchange there is a limit of 1000 messages/sec, so it means that if HFT company burst whole 1000 of messages in first half of the period, in second half it cannot send any message, so it’s cut out of the market. Other limitation used by exchanges is a limit of messages per trade. It hits even harder in quote stuffing and spoofing. In most of the cases limit is around 500 messages per trade and if someone exceed it then he should be prepared for fines. On top of it company that frequently break limits could be banned from exchange for some time.
If we talk about front running, first thing we have to know is a fact that front running, in the dictionary meaning, is illegal action, and there are big fines for caught market participants who use it. Front running is using informations about new orders before they will go to the order book. Let’s say Broker gets new order with price limit to process, but before putting it to exchange, he will buy all available shares at better price than limit and then he execute client’s new order at limit getting extra profit. That’s highly not allowed and that’s not what HFT companies do.
All they do is tracking data feed, analyzing quotes, trades, statistics and basing on that information they try to predict what is going to happen in next seconds. Of course, they have advantage due to latency on data feed and so on, because of co-location, better connection and algorithms, but it’s still fair.
HFT companies have to play on the same rules as other market participants, so they don’t have any special permits letting them do things not allowed for others. Same with Dark Pools, specially that they are regularly controlled by Finance Regulators.
First, we have to know that suppliers of liquidity, i.e. Market Makers and some investors use HFT. They place orders on both sides of the book, and all the time are exposed to sudden market movement against them. The sooner such investors will be able to respond to changes in the market, the more he will be willing to place orders and will accept the narrower spreads. For market makers the greatest threat is the inability to quickly respond to the changing market situation and the fact that someone else could realize their late orders.
System performance in this case is a risk management tool. Investments in the infrastructure, both a software and hardware (including co-location), are able to improve their situation in terms of risk profile. The increase in speed is then long-term positive qualitative impact on the entire market, because it leads to narrowing of the spread between bids and offers – that is, reduce the transaction costs for other market participants, and increase of the liquidity of the instruments.
HFT AND MARKET QUALITY
In April of 2012. IIROC (Investment Industry Regulatory Organization of Canada), the Canadian regulatory body, has changed fee structure based so far only on the volume of transactions, adding the tariffs and fees that also take into account the number of sent messages (new orders, modifications and cancellations). In result, introducing new fees made trading in the high frequencies more difficult. It was very clearly illustrated by data from the Canadian market.
Directly in the following months these fees caused a decrease in the number of messages sent by market participants by 30% and hit, as you might guess, precisely the institutions that use high-frequency trading, including market makers. The consequence for the whole market was increase in the average bid-ask spread by 9%.
NO PLACE FOR MISTAKES
When people talk about HFT, both enthusiast and critics, it is not rare to hear that HFT is risk free. Well, on the face of it, after analyzing how HFT works you would possibly agree with it, but there is a dangerous side of HFT that can be not so obvious and people often forgot about it. HFT algorithms works great if the code is well written, but what would happen if someone would run wrong, badly tested or incompatible code on a real market?
We don’t have to guess it, because it happened once and it failed spectacularly, it was a “Knightmare”. Week before unfortunate 1st of August Knight Capital started to upload new version of its proprietary software to eight of their servers. However Knight’s technicians didn’t copy the new code to one of eight servers. When the market started at 9:30 AM and all 8 server was run, the horror has begun. Old incompatible code messed up with the new one and Knight Capital initiated to lose over $170,000 every second.
It was going for 45 minutes before someone managed to turn off the system. For this period Knight Capital lost around $460 million and became bankrupt. That was valuable lesson for all market participants that there is no place for mistakes in HFT ecosystem, because even you can gain a lot of money fast, you can lose more even faster.
HFT is a natural result of the evolution of financial markets and the development of technology. Companies that invest their own money in technology in order to take advantage of market inefficiencies deserve to profit like any other market participant.
HFT is not as black as is painted.
Aldridge, Irene (2013), High-Frequency Trading: A Practical Guide to Algorithmic Strategies and Trading Systems, 2nd edition, Wiley,
Empirica’s Algorithmic Trading Platform has successfully passed the XDP protocol communication certification, issued by the Warsaw Stock Exchange.
From now on Empirica is officially listed as the ISV (Independent Software Vendor) for the Warsaw Stock Exchange.
WSE uses Universal Trading Platform delivered by NYSE Technologies. The same system is used by many other European and world stock exchanges. Fulfilment of technical criteria of Warsaw Stock Exchange makes certification for those markets only a formality for our platform.
Time-Weighted Average Price (TWAP) is another trading algorithm based on weighted average price and in compare to Volume-Weighted Average Price its calculations are even simplier. Also it’s one of the first execution algorithms and unlike most algorithms nowadays it’s passive execution algorithm that waits for proper market price to come, doesn’t chase it.
As TWAP doesn’t bother about volume it’s extremely simple to obtain it. All it takes is to get Typical Price for every period bar using equation below and then calculate average of Typical Prices.
Typical Price = (Close+High+Low+Open)/4
Let’s just take a look at example results calculated on 1-minute interval intraday Morgan Stanley’s stock.
The most common use of TWAP is for distributing big orders throughout the trading day. For example let’s say you want to buy 100,000 shares of Morgan Stanley. Putting one such a big order would vastly impact the market and the price most likely would start to raise. To prevent that, investor can define time period in TWAP Strategy over which they want to buy shares. It will slice evenly big order into smaller ones and execute them over defined period.
TWAP could be used as alternative to VWAP, but because of itssimplicity we have to remember about some pitfalls. Even if we slice big orders, we do it evenly, thus there is a possibility to hit on low liquidity period when our splitted order will impact the market hard. That’s why it’s recommended to use TWAP over short periods or on stocks that are believed to not have any volume profile to follow.
There is also another threat coming directly from dividing big order evenly, namely, other traders or predatory algorithms. Obviously trading in such a predictable way can lead to situation where other traders or algorithms would look through our strategy and start to “game” us.
Barry Johnson in his book suggests adding some randomness to the strategy as a solution to the issue. He says that “We can use the linear nature of the target completion profile to adopt a more flexible trading approach. At any given time, we can determine the target quantity the order should have achieve just by looking up the corresponding value on the completion rate chart.”
In practice it means that when we have run 4-hour TWAP we don’t slice the order into evenly parts, but otherwise we focus on percentage completion. So for instance we would want to have 25% of the strategy completed by first hour, 50% by second and 75% by third. That gives a more freedom into size of orders, so we can be more random with it and hence less predictable for other traders on the market.
TWAP vs VWAP
As both indicators use same mechanism, i.e. weighted average price, it’s common to compare them. Despite that VWAP’s nature is more complex and includes volume in its calculations, on instruments with low turnover TWAP and VWAP values can be close. On the other hand when a session starts to be more volatile both indicators will diverge.
On a table below there are TWAP and VWAP calculated for whole trading day. As we can see at the beginning of the trading day the difference is less than a cent, but on close the difference raised up to 2 cents. It happened because during the day there were some small volume trades for lower price that didn’t affected VWAP, but did TWAP.
TWAP Strategy is another great tool for executing big orders without impacting the market too hard. Like everything it has its own pros and cons and it’s up to us to select if TWAP will be the best strategy to use for our case or maybe we should consider using VWAP or other strategy.
- H. Kent Baker, Greg Filbeck. “Portfolio Theory of Management” (2013) , pp.421
- Barry Johnson “Algorithmic & Trading DMA – An introduction to direct access trading strategies” (2010), pp. 123-126
Nanex released a video showing the results of half a second of worldwide high frequency trading with Johnson and Johnson stock. I simply sped up the footage to get a better feel of what it looked like. Blow Your Mind.
CNN’s Maggie Lake gets a rare look inside the super-fast trading industry.
Dr. Sean Gourley is the founder and CTO of Quid. He is a Physicist by training and has studied the mathematical patterns of war and terrorism. He is building tools to augment human intelligence.
Citadel Group, a high-frequency trading firm located in Chicago, trades more stocks each day than the floor of the NYSE.
One of the scariest high frequency trading algos ran in the electronic S&P 500 futures (eMini) contract on January 14, 2008 starting at 2:01:11Eastern. During its 7 second reign, there were over 7,000 trades (52,000 contracts), and the price eventually oscillated within milliseconds, the equivalent of about 400 points in the Dow Jones Industrial Average!
HFT is controlled by proprietary trading firms and spans across multiple securities, including equities, derivatives, index funds and ETFs, currencies and fixed income instruments.For HFT, participants want the following infrastructure in place:
– High speed computers, which need costly and regular hardware upgrades;
– Real time data feeds, which must avert even the delay which could affect profits; and of a microsecond
– Computer algorithms, which are the heart of HFT and AT.
Benefits of HFT
– HFT is beneficial to traders, but does it help the total marketplace? Some market that is overall gains that HFT assistants cite contain:
– Bid-ask spreads have reduced due to HFT trading, making markets more efficient. Empiric evidence contains that after Canadian authorities in April 2012 imposed fees that deterred HFT, studies indicated that “the bid-ask spread rose by 9%,” possibly due to diminishing HFT trades. And thus facilitates the effects of market fragmentation.
– HFT assists in the price discovery and price formation process, as it is centered on a high number of orders (see related: How The Retail Investor Profits From High Frequency Trading.)
Algorithmic Game Theory and Practice, Michael Kearns, University of Pennsylvania,
Traditional financial markets have undergone rapid technological change due to increased automation and the introduction of new mechanisms. Such changes have brought with them challenging new problems in algorithmic trading, many of which invite a machine learning approach. I will briefly survey several algorithmic trading problems, focusing on their novel ML and strategic aspects, including limiting market impact, dealing with censored data, and incorporating risk considerations.
Harnessing the power of machine learning for money making algo strategies with Bert Mouler
The following frontier of the technological arms race in finance is artificial intelligence. Improvements in AI research have triggered massive curiosity about the sector, where some consider a a trading, learning and thinking computer will make even today’s superfast, ultra-complicated investment algorithms appear archaic — and potentially leave human fund managers redundant. Could the next generation’s Buffett be a super-algo?
Some of the world’s largest cash managers are betting on it. AI investing may sound as, although fantastical sci-fi writer William Gibson said The future is already here, it’s simply not evenly dispersed.” Bridgewater, the world’s biggest hedge fund group, poached the head of IBM’s artificial intelligence unit Watson in 2012, and Two Sigma and last year BlackRock, another rapidly growing hedge fund that uses quantitative models, hired two former top Google engineers. Headhunters say computer scientists are now the hottest property in finance.
The quantitative investment world plays down the prospect of machines arguing that human genius still plays an important part, pointing out that the prospect of artificial intelligence that is complete is still distant, and supplanting human fund managers. But the confident swagger of the cash management nerds is unmistakable. There are quasi-AI trading strategies working their magic and the future belongs to them, they predict.
Artificial intelligence and video that is financePlay
The time will come that no human investment manager will manage to defeat the computer.” Or, as Agent Smith put it succinctly in The Matrix: “Never send a human to do a machine’s occupation.”
Yin Luo first learned to code after an used Apple II computer was brought back to China by his dad from a business trip to West Germany in 1985, when he was 11. But there were no games to purchase his home town in Heilongjiang province, in Yichun, so he made a crude variation of Tanks, where the player shoots down randomly created aeroplanes and taught himself to program.
It was arduous work. The computer’s lack of memory meant it crashed the program coding grew not too simple. He’d no floppy discs, so he learnt how to save the info on cassette tapes. “I just really needed something to play with,” Mr Luo recalls.
But the expertise paid off. Now, he is part of a growing tribe of brainiacs on Wall Street investigating the bleeding edge of computer science.
A network of 20 Linux servers is needed to run the hyper-sensible “ adaptive style turning” that was linear model, which is founded on a “machine learning” algorithm.
Machine learning is a branch of AI a diffuse term that is certainly frequently misused or misunderstood. While many people comprehend AI to mean sentient computers like the archvillains SkyNet in the Terminator films or HAL 9000 in 2001: A Space Odyssey, in practice everyday tools including Google’s language translation service, Netflix’s film recommendation engine or Apple’s Siri virtual assistant install basic forms of AI.
Quants have long used increasingly powerful computers to crunch numbers and uncover statistical signs of money-making opportunities, but machine learning goes a step further.
It can learn the difference between apples and bananas and sort out them, or perhaps instruct a computer how to play and quickly master a game like Super Mario from scratch. Machine learning can also be unleashed on “unstructured data”, such as for example jumbled amounts but also pictures and videos which can be typically not easy for a computer to comprehend.
More powerful computers mean that it are now able to be applied to financial markets, although the technique is old. “It’s a very bright area,” Mr Luo says. “Artificial intelligence is able to help you find designs an individual would never see. That may give you an enormous advantage.”
But that is not the only advantage of machine learning.
When marketplaces undergo what industry participants call a regime change that is “ ” and trusted strategies no more use, one of the classic challenges for quants is that their models can often prove worthless — or worse. Algorithmic trading strategies that print cash one day can blow the next up.
A machine learning algorithm adjusting to what works in markets that day, will autonomously develop and search for new patterns.
That means they can be used by asset managers as something to develop trade and strategies by itself, or perhaps to enhance their investment process, maybe by screening for patterns undetectable by people.
For Nick Granger, a fund manager at Man AHL, a quant hedge fund, that is the advantage that is critical. “You see it creating intuitive trading strategies in the bottom up, changing styles according to what works,” he says. “ We have been using machine learning for the past few years and have an interest in investing it in more.”
Nonetheless, machine learning has pitfalls. One of the largest challenges for quants is a phenomenon called “overfitting”, when an algorithm that is coded or exceedingly complicated finds false signals or specious correlations in the noise of data. For instance, a blog called “Spurious Correlations” notes that margarine consumption is linked to Nick Cage pictures to swimming pool drownings, and divorce rates in Maine.
When confronted by actual markets even if your model functions well in testing it can fail. Also, new data can be changed by the trading algorithm, says Osman Ali, a quant at Goldman Sachs’ asset management arm. “ you’re not affecting the weather, but if you deal marketplaces they are being affected by you If you crunch weather data.”
Nor can the most complex AI think as creatively as an individual, especially in a crisis. Brad Betts, a former Nasa computer scientist now working at BlackRock’s “ active equity” arm that is scientific, emphasizes the 2009 emergency plane landing on the Hudson river by Chesley Sullenberger of when machine is trumped by man as an example.
Truly, some quants remain sceptical that machine learning — AI or more broadly — is a holy grail for investment. Many see it just as a fresh, sophisticated gizmo to supplement their present toolkit, but others claim it truly is mainly a case of intelligent marketing rather than something genuinely ground-breaking.
They do’t constantly work, although “People are always desperate to find new ways to earn money in financial markets. He points out the human brain is uniquely adept at pattern recognition, “whether it really is love, a triangle or a face. Investment management is totally amenable to being addressed by computers designed to see patterns, but I’m not going to rush to use the latest hot algo to do so.”
Algo-trading provides these advantages:
Algorithmic Trading Strategies
Educational material for basics of algorithmic trading
Algorithmic Trading– Impact of Automated Trading Programs On Markets Documentary
Learn about the impact of automated trading systems on today’s markets. While this documentary focuses on stocks, the same factors are at work in the Forex markets. High frequency, algorithmic trading programs work quickly and create huge volatility. This excellent documentary Money & Speed is from VPRO which is required viewing for all traders.
TEDxConcordia – Yan Ohayon – The Impact of Algorithmic Trading
Yan Ohayon demystifies and shares his experience with algorithmic trading and its impact on markets, our lives, and everything in between.
Quants: The Alchemists of Wall Street – A Documentary about algorythmic trading
For news on cryptocurrencies, algorithmic trading, liquidity, market making, arbitrage, bitcoin, ethereum, crypto exchanges, quantitative cryptocurrency strategies, blockchain and algorithmic trading software.
© Copyright 2010-2018
Empirica, the vendor of
Cryptocurrency Algorithmic Trading Software