Exchange Simulators for SOR / Algo Testing: Advantages vs. Shortcomings

Alexey Zverev, Alyona Bulda

Exactpro Systems LLC

SOR & Other Trading Algorithms

Introduction

Examples

Quoting and Market Making Executing substantial orders automatically Smart Order Routing Arbitrage

SOR & Other Trading Algorithms Abstract Scheme

Testing of SOR & Trading Algorithms

SOR or Algo Trading systems: 

Trade automatically

May or may not receive external instructions (parent orders)

Connects to one or multiple execution venues (markets, exchanges)

Connects to low latency Market Data Source to receive current prices

Makes a decision based on market data and statuses of the orders sent to the external markets. 

Testing of SOR & Trading Algorithms

System Testing

• Algo developers need an ability to verify the correctness of algorithm implementation within the trading platform

• Testing of surrounding infrastructure (module testing)

Example order should be delivered correctly between algo and Market

Algo Performance Testing / Back testing

• Analysts need an ability to test the algorithm profitability

This is usually done via back testing (applying algo to historical data) or applying algo against some model simulating market behavior or combination of these approaches

Testing of Trading Systems Connected to Electronic Markets

All electronic markets provide test environments where clients are able to test their connectivity. These environments are supposed to be identical to production environment.

Alternative to testing against such test environment is using a Test Harness. 

Testing of Trading Systems Connected to Electronic Markets

Trading system connectivity layer usually has a specific layer isolating other components from the specific Market API implementation

Exchange Emulator can either emulate a specific API or an intermediate component. This way we can reduce the amount of API development work for our Test Harness

Exchange Simulators: Their Role & Species

Exchange Simulators is something that simulates the market in order to test trading systems

 There are two types Exchange Simulators:

API Emulators – they emulate exchange connectivity, provide scriptable responses and unsolicited events

Market Simulators – they simulate market/ multiple markets responses based on some model

 The API emulator is used to emulate external exchange connection in a controlled manner

The Market Simulator is for testing algorithms via exchange simulations or historical market data replay (back testing)

SOR & Other Trading Algorithms vs. Exchange Simulators: Abstract Scheme

The Advantages of Using Exchange Simulators

Exchange Simulators:

Provide more control over testing

Enhance the ability to emulate various events to increase test coverage

Provide the ability to test the algo incorporated into the real production software (not in Excel or Matlab)

 ... under the real load mimic the real production environments 

The Shortcomings of Using Exchange Simulators, API Emulation

An Exchange is a complex proprietary black box system. Its nearly impossible to emulate its API behavior accurately

 This black box constantly changes

Algo Trading Systems usually have to deal with multiple Exchanges

In a High Frequency Trading network, delays and hardware issues matter. This cannot be simulated 

The certification by the Exchanges of the Gateways connecting to the Exchanges should happen in Exchange Test Connection, so API emulator development is duplication of effort 

The Shortcomings of Using Exchange Simulators, API Emulation

Real life examples:

 The other side of the barricade is supporting client certification of a trading venue. The following issues could be discovered only via the certification process:

Missing Execution Reports recovery through resend request mechanism

XML instrument definition distribution

Exotic multi-leg trades 

Runtime instrument creation

Implied liquidity dissemination

The Shortcomings of Using Exchange Simulators, Market Simulation

It is impossible to develop a Market Model, only some localized bits and pieces

The Market Model is a valuable intellectual property of buy side players. Information is not available for public research

Requires a lot of real trading / HFT experience to be able to come up with it

An Exchange Simulator must be really flexible in terms of its model

Dark/Hidden Liquidity Markets are impossible to analyze and predict

The Development of such a system requires a tremendous effort in terms of development and financial math resources and costs a lot; and anyway is far worse than the real market in terms of the development of real Algos

The Shortcomings of Using Exchange Simulators, Market Simulation

Real life example: the Exactpro Exchange Simulator

The Shortcomings of Using Exchange Simulators, Market Simulation

Exchange Simulator Design:

 Order Book exactly replicating all exchange rules

This is already a challenge – real life example

Trading Systems connect to the simulator as if they are connecting to the real exchange (both for OE and MD)

Need to support adequate load generated by Trading Algos

Market Model is implemented in three instances:

The system is fed with historical orders recreating historical order books 

A bunch of competing strategies representing the real market

Counter flow model constantly monitors the liquidity on the book and generates artificial events to simulate market impact of orders generated by the Trading System under test