SmartQuant Discussion

Simulation (Execution) provider uses bar data to fill orders when FillOnBar option is set to true. When a solution uses 2+ timeframes it is recommended to setup Simulation (Execution) to use only bars from the shortest timeframe. Otherwise the backtesting results may become less realistic.

For example:
System uses two timeframes - 1min and 1hour bars. Let's consider time interval from 10:00 till 11:00. The lowest price is $100 over this interval, this price occured only once at 10:15. At 10:40 current price is 120$ and the system decides to put a buy Limit order at $101 level. The price never falls down to this level till 11:00, so this Limit order should not get filled. But when 1 hour bar (10:00-11:00 bar) is generated the Limit order gets filled by Simulation (Execution) provider since the Low of this bar is $100 which is less than the Limit level. If Simulation (Execution) provider used only 1 min bars to fill the orders this kind of situation would never happen.

To add/remove bar timeframes that can be used by Simulation (Execution) to fill orders - go to the properties of Simulation (Execution) provider and:
- set BarFilter.Enabled property to true.
- add/remove timeframes using BarFilter.Items property. Simulation (Execution) will use only timeframes listed in the Items property to fill orders.

Thank you for this informative post.

I'm having issues with realisticness of backtesting execution on 1h bars.
I'd like feed the Simulator with 1mn bars too so that it knows whether the high or the low was first and triggers orders accordingly, in a realistic fashion.
Note that I'm not working on a multi-timeframe strategy at all, I just want to improve the execution assumptions to get closer to reality.
When I enable the BarFilter and add Bars Time(60) (Enabled) to its Items property, the OnBar() event is called both for 1mn and 1h bars although I only want to use 1h bars in my code.
I can filter out those bars in the OnBar() event if Bar.Duration.Minutes != 60 by calling "return;".

However, my custom indicators are instanciated in OnStrategyStart() and they are constructed from Strategy.Bars (as the first argument). How do I know which bars are being used to compute my indicators values (1mn or 1h)? How do I force them to use 1h bars?

Thanks!

Hi,

You should use GetBars method instead of Bars property when creating your indicators:

This way you specify the exact bartype and barsize of the input series.

Regards,
Sergey.

Hi



Good question PBJ. I wonder too.



Thank you Sergey for information

Regards
MaRCHeW

_________________
"Your worst day with OpenQuant is better than your best day with most of the other stuff."

Thanks Alexei. Now my 2 custom indicators are constructed using HourlyBars that I define in OnStrategyStart() as HourlyBars = GetBars(BarType.Time, 3600);

That said, here's OpenQuant.exe footprint when I start OQ and then run a backtest using 1h + 1mn data, with BarFilter enabled and BarFilter.Items[0] = Bars Time(60).
CPU on top (4 cores), memory consumption in the middle, I/O at the bottom.



Backtesting on 1mn data for 1 instrument takes far more time than on 1h bars but the way OQ's backtesting engine works explains why.
However, memory consumption is clearly HUGE and almost always increasing either slowly or by 'gapping'.
I have only 2 custom indicators instanciated once using hourly bars and they have no specific memory usage: I can see that when I backtest using only hourly bars.
Also, memory is not released after the backtest (I can understand that, it's a design choice).

What I don't get is why OQ reads so much IO and why it consumes so much memory.
Even with ALL 1mn bars (count = 3 million) for this instrument fetched in memory (either at once at the beginning or progressively), this data is 152 MB in a CSV file and 18 MB zipped. I assume the compression algo used in your binary DB makes the data closer to 18 MB than 152 MB but anyway, even if it was 500 MB, it doesn't explain why OQ uses only ~100 MB of RAM at the beginning and then 1.5 GB at the end of the backtest.

Hi, have you looked through this FAQ? Have upi applied any constraints?

Also, how many orders do you generate during backtest? Have you set an option to remove done orders from the order manager? Performance curves calculation options? A backtest generates much more data than you have in your original bar series...

And also don't forget that .net simply doubles collection size when it adjusts max number of elements stored in a collection. I think "steps" that you can see on the memory consumption curve on your graph show just this. So that collections can be almost half empty at the end of backtest, i.e. real data take just 700M and another 700M of memory is reserved by .net collections.

Thanks Anton for your reply. Your post already gives me more info.
Here are more details...

I have read the FAQ extensively and don't see how it can help further.
What do you call "constraints"?

Setting Trade Array and Quote Array Sizes to 1 instead of -1 pretty much doesn't change anything (RAM consumed seems closer to 1.5 than 1.7 but that's could be due to other factors).

At the end of the 8-years backtest, whether Trades Array Size and Quote Array Size = -1 or 1:
3162 transactions were generated (Portfolio.Transactions.Count).
7164 orders objects exist in Orders (Orders.Count).

I had not checked the option to remove done orders from the orders manager and Performance curves & drawdowns were calculated (default options).

When I check the option to remove done orders from the orders manager, a call to CancelOrder() results in an exception.

However, when I uncheck the options to calculate performance curves & drawdowns, the backtest runs faster and memory usage remains much lower!
This seems to be a genuine good tip for anyone looking to improve backtesting performance.
Memory simply goes up in a linear fashion when I start the backtest and stops when the backtest is done. It goes from 55 MB (just after starting OQ) to 268 MB (end of backtest) instead of 1.5-1.7 GB.
This is a very good step in the right direction.
I had anyway planned to dismiss OQ's performance calculations later on and build my own performance analyser/portfolio builder (persist trades to a DB, perform queries on it, extract P&L and other perf indicators either just after the backtest or in a later stage => more flexibility).

FYI, the exception related to Order manager: