SmartQuant Discussion

Automated Quantitative Strategy Development, SmartQuant Product Discussion and Technical Support Forums
It is currently Sun Sep 24, 2017 6:02 am

All times are UTC + 3 hours




Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: OpenQuant 2014
PostPosted: Thu Jan 02, 2014 5:15 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
We are pleased to announce the launch of completely redesigned SmartQuant framework and OpenQuant 2014 IDE.

You are welcome to contact us at info@smartquant.com to get a copy of OpenQuant 2014 preview and to take part in the beta testing program.

Introduction to OpenQuant 2014 for OpenQuant Users: http://www.smartquant.com/openquant2014/doc/Introduction%20to%20OpenQuant%202014%20for%20OpenQuant%20users.pdf

Happy New Year!

Image

The framework has been re-designed and re-programmed from scratch. There can be many frameworks running in parallel within one application on the same computer. The framework (API) is open to the end user. You can use it to create your own trading applications.

A new architecture based on the event bus concept is the core of the new framework. This new architecture allows us to efficiently utilize multithreading, caching, balance workload and construct distributed trading applications.

Simulation speed has been increased dramatically. Parallel strategy optimization can now be done in several ways, either by running several isolated framework instances in parallel or by distributing simulator events to several frameworks connected via event bus.

Thanks to the event bus and event pipes concept, dynamic market data subscriptions during simulations are possible now. This was not possible before. This feature open doors to a large subset of strategies, such as automatic futures rollovers, option chains, dynamic portfolio allocation and rebalancing, etc.

Multiple providers are supported. Providers call strategy event handlers from the same thread. Actually all strategy event handlers are called from the same thread now and this resolves a great number of conflicts that OpenQuant users had before, especially in the real time mode or while using/developing their own custom providers.

Framework clock can rely on computer hardware and trigger timers (remainders) with nanosecond precision.

Data base engine has been rewritten from scratch. Multiple clients accessing data file on local host or via network are now supported through data file server service. Insert and delete operations are much faster now. The data engine can store any named objects, such as instruments or portfolios, which are stored now in the data file instead of slow MS Access database.

Execution layer has been rewritten from scratch. Instead of slow FIX messages the new OpenQuant uses fast and compact execution messages, which have a set of standard predefined fields, but can also have a set of user defined fields. This significantly speeds up simulations and minimizes memory consumption, so that now OpenQuant is able to simulate millions of orders in pretty reasonable time.

Execution messages are two-way messages now (from and to a broker). They are either execution commands or execution reports, so that it’s possible now to keep and track the entire step by step execution history associated with an order.

Hierarchies of strategies and portfolios become possible. A strategy can have child strategies and a portfolio can have a parent portfolio. There is no need for a special metastaregy class any longer – every strategy can be a metastrategy and you can build trees of strategies of different types. For example this means that now it’s possible to see not only aggregated, but also per instrument portfolio and performance statistics for OpenQuant InstrumentStrategy.

OpenQuant 2014 solutions are normal Microsoft Visual Studio solutions, which can be opened, compiled, debugged and executed in Visual Studio.

There are several strategy types built into framework. Strategies derived from the base Strategy class receive events for all instruments, while InstrumentStrategy creates an instance per instrument internally. ComponentStrategy works with Signals propagating through a set of strategy components such as DataStrategy, AlphaStrategy, RiskStrategy, PositionStrategy, ExecutionStrategy, ReportStrategy.

New class of Sell Side Strategies allows to support truly synthetic instruments (pairs, spreads, baskets, etc.), which are seen as normal instruments to the rest of framework and to Buy Side Strategies; to implement a variety of broker side strategies, such as algorithmic orders, automatic rollovers, etc.

A completely new strategy logging and monitoring system was designed and developed.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Thu Jan 23, 2014 12:14 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
Multicore optimization and many other features have been added to OpenQuant 2014 beta since initial realease. You can download updated version using the same download link. Don't foget to do clean uninstall as discussed in the introductory doc:

Reinstalling

Since OpenQuant 2014 is an early beta, which we update quite frequently, it’s a good idea to perform “clean” uninstall before installing a new version. OpenQuant installer doesn’t override user files such as data files, configuration files or strategy files. Thus you may miss important changes in those files if you don’t remove them manually.
In order to perform “clean” uninstall, you should uninstall OpenQuant selecting Uninstall in windows start menu,

and then manually remove files from your AppData\Roaming\SmartQuant Ltd\OpenQuant 2014\ and your Documents\OpenQuant 2014\Solutions\ folders . Indeed you can leave or backup files that you need. For example you don’t need to delete your own strategies if you need them, but if you want to update demo strategies, you need to delete corresponding files.
Then you can install a new version of OpenQuant 2014 as discussed in the previous chapter.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Tue Jan 28, 2014 9:21 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
The database engine was heavily refactored and should be even more speedy and reliable now. The raw format of data files was changed and you need to perform clean uninstall in order to successfully upgrade to this version from any previous version.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Mon Feb 03, 2014 6:43 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
A number of new features was added, particularly:

Using external series in the data simulator

The DataSimulator can use external data series to simulate the flow of market data events stored in these series. You can add external data series to the DataSimulator using DataSimulator.Series.Add() method.

This feature can be used to significantly speed up strategy simulations. The idea is to run strategy simulations normally for the first time and record data event flow into a series in a data file. Next time you can run strategy simulations using this data series instead of usual strategy market data subscriptions. When you run strategy simulations in a normal way, data simulator adds a number of event queues into the data pipe of the event bus. Actually it adds one event queue per market data request per instrument, so that if you have bid, ask and trade series per instrument and add hundred instruments into a strategy, the data pipe is going to consists of three hundred event queues (one event queue per simulated data series). Consequently, events from these queues should be correlated before they can be used to trigger strategy event handlers. This event correlation process can take a lot of CPU resources (in the most simple and naive approach we should look up all three hundred queues and find an event with the most recent datetime every time we need to pick up the next strategy event). When we use DataSeriesEventLogger to record data event flow coming into our strategy, we record a flow of already correlated events, so that we don’t need to spend time to correlate events if we run the data simulator next time with such pre-recorded data series.

The process of making bars from ticks in the BarFactory for a number of time frames and instruments can take significant CPU time as well. Again, you can use technique discussed above to simulate pre-recorded series of bars.

Using data series cache to speed up simulations

Data series objects can be cached in memory when you read data series from disk for the first time. This means that next time you access the same data series object, it will be almost immediately returned from memory. Data series objects will not be loaded from disk, decompressed and deserialized. This can significantly speed up repeating simulations. Indeed you should have enough free memory installed in your computer to keep all simulation data in memory cache.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Mon Apr 07, 2014 8:55 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
New OpenQuant 2014 update includes several samples (MS Visual Studio solutions with full source code) of user data/execution providers (OpenECry), indicators and a demo application (trading console with algo orders) that can be developed with SmartQuant Framework.

Image


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Sat Apr 12, 2014 11:25 am 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
We are pleased to announce that OpenQuant 2014 beta http://www.smartquant.com/openquant2014.html goes public.

Download OpenQuant 2014 public beta from http://www.smartquant.com/download_openquant2014.html and send us your comments and suggestions or discuss in OpenQuant 2014 section of SmartQuant public forum at http://www.smartquant.com/forums/viewforum.php?f=86.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Tue May 20, 2014 10:07 am 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
New OpenQuant 2014 release includes new portfolio statistics engine. Now it is possible to use portfolio statistics in strategies and develop user defined portfolio statistics. OpenQuant 2014 install comes with Sample Statistics MSVS project that shows how to develop and use custom statistics.

Image


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Thu May 22, 2014 9:32 am 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
Now you can use two chart widgets in OpenQuant 2014:

index based chart (gapeless)

Image

and time based chart

Image

Both chart types have their pros and cons and you can select the most suitable type for your studies. You can also have both charts at the same time within the same OpenQuant 2014 application.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Sat Sep 27, 2014 2:35 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
IQFeed adapter is included in the latest OpenQuant 2014 setup.

Regards,
Anton


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Tue Nov 04, 2014 2:30 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
QuantFileServer has been significantly redesigned. Now it's possible to use the server with several clients reading from/writing to data series at the same time. This means that QuantFileServer can replicate QuantBase in OQ2014 world and provide even more functionalities to the end users. Now you can truly have one central and remote data server that can be used by everyone for backtesting, feed capture, historical data storage and so on.

Regards,
Anton


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Mon Nov 17, 2014 2:33 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
Using TimeSeries with data stored in memory and on disk

TimeSeries use IDataSeries objects internally to store time series data entries (TimeSeriesItem). There are two implementations of IDataSeries interface in the SmartQuant framework: MemorySeries and DataSeries.
Usually TimeSeries use MemorySeries to store data points in memory. TimeSeries default constructor creates MemorySeries behind the scene, but you can also pass DataSeries to TimeSeries constructor to work with data stored in OpenQuant database.
DataSeries dataSeries = DataManager.AddDataSeries("MyDataSeries");

TimeSeries series = new TimeSeries(dataSeries);

series.Add(new DateTime(2014, 11, 17), 1);
series.Add(new DateTime(2014, 11, 18), 2);
series.Add(new DateTime(2014, 11, 19), 3);

and then
DataSeries dataSeries = DataManager.GetDataSeries("MyDataSeries");

TimeSeries series = new TimeSeries(dataSeries);

for (int i = 0; i < series.Count; i++)
Console.WriteLine(series.GetDateTime(i) + " " + series[i]);

This way you can use time series with data stored on disk as if they were stored in memory.

Using this feature
• You can use common TimeSeries interface to work with data in the database
• Your time series become persistent (any change in time series is automatically reflected in the database)
• You can process very large amounts of data, working with time series that normally cannot fit into computer memory
The DataManager provides two helper methods DataManager.AddTimeSeries() and GetTimeSeries() that you can use to simplify the code above
TimeSeries series = DataManager.AddTimeSeries("MyDataSeries");

series.Add(new DateTime(2014, 11, 17), 1);
series.Add(new DateTime(2014, 11, 18), 2);
series.Add(new DateTime(2014, 11, 19), 3);

and then
TimeSeries series = DataManager.GetTimeSeries("MyDataSeries");

for (int i = 0; i < series.Count; i++)
Console.WriteLine(series.GetDateTime(i) + " " + series[i]);


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Tue Dec 30, 2014 4:31 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
Controlling running strategies (changing parameters and calling methods)

You can change parameters (fields or properties) or call methods of running strategies on the fly either directly in OpenQuant IDE or remotely using QuantController / QuantConsole.
Strategy properties that you want to change should be marked with [Parameter] attribute in the strategy code, for example in the case of Bollinger Bands demo strategy:

Code:
      [Parameter]
      public double AllocationPerInstrument = 100000;

      [Parameter]
      public double Qty = 100;

      [Parameter]
      public int Length = 10;

      [Parameter]
      public double K = 2;

Once your parameters are marked with [Parameter] attribute and your strategy is running, you can open Strategy Manager Window in OpenQuant IDE to see the tree or running strategies.

You can click on a particular strategy and observe / change strategy parameters in the Properties Window

Let’s change Qty parameter from 100 to 500 for AAPL strategy instance. We get the following message in the Output Window confirming that Qty parameter has been successfully changed.

Remember that when you run an InstrumentStrategy, you actually run a strategy tree with a parent strategy holding strategy instance per instrument. This is exactly what we see in the Strategy Parameters window – a tree of the BollingerBands instrument strategy with two strategy instances for AAPL and MSFT instruments.

Note that we can change parameters for a particular strategy instance / instrument. In our example we change Qty for AAPL instrument (Qty = 500), but it stays the same (Qty = 100) for MSFT instrument. If you want to change particular parameter for all instances simultaneously, you can select the parent strategy (BollingerBands root strategy in our case).

Let’s change Qty parameter to 250 for all instrument instances. The Output Window suggests that we have changed Qty parameter in the parent strategy and in both instrument instances, AAPL and MSFT.

Note that when you change strategy parameters, you only change values of corresponding fields or call setters for properties with explicit setters. This means that if you change K or Length parameters in our BollingerBands demo example, BBU/BBL indicators will not be actually affected since initial values of these parameters has already been passed to BBU/BBL indicator constructors at indicator creation time, and apart from this point, there is no connection between K or Length field and indicator objects. You should do some extra work to pass new values of K and Length parameters to indicators if you really want your strategy to behave this way. On the other hand if you change Qty property, all new strategy orders will be send with this updated quantity.

You can also mark certain methods in your strategy class with [StrategyMethod] attribute. Note that such methods should take no parameters. An obvious example of such callable method is the Panic method of your strategy that usually closes all outstanding orders and halts future trading. In our tutorial we simply write Panic in this method to demonstrate how this concept works:

Code:
      [StrategyMethod]
      public void Panic()
      {
         Console.WriteLine("Panic " + Instrument);
      }


Once you mark a method with [StrategyMethod] attribute and run your strategy, the name of this method will appear in the Strategy Monitor window under corresponding strategy folder:

If you double click on the method name, this method will be called in the corresponding strategy:

Similarly to parameter settings in the InstrumentStrategy, a method will be called for all strategy instrument instances if you double click on the parent strategy method name:

You can find this text with screenshots in the Developing Trading Strategies with OpenQuant 2014 doc.

Happy New Year!
Anton


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Thu Jan 22, 2015 2:18 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
How to control event bus latency in the real time mode

The Event Dispatcher of the SmartQuant Framework constantly tries to dequeue an event (market data, execution report and so on) from the Event Bus and pass it to your trading strategy or application. If the Event Bus contains no events, the bus turns into the idle mode. Such idle mode can be organized in several ways. The event bus thread can go to sleep for a while, it can spin until it has an event to dequeue, or it can wait until it gets a notification about a new event added to the bus. This behavior can be controlled with the EventBus.IdleMode option, which can be Spin, Sleep or Wait.

EventBusIdleMode.Spin adds almost no latency but it eats 100% of one core of your CPU since it constantly checks for a new event in [while empty do nothing] loop.

EventBusIdleMode.Sleep calls Thread.Sleep(1) and adds 1ms latency if you have a stream of rare events, but on the other hand it almost completely releases the CPU.

EventBusIdleMode.Wait employs ManualResetEventSlim . As MSDN suggests, ManualResetEventSlim can be used for better performance when wait times are expected to be very short, and when the event does not cross a process boundary. ManualResetEventSlim uses busy spinning for a short time while it waits for the event to become signaled. When wait times are short, spinning can be much less expensive than waiting by using wait handles. However, if the event does not become signaled within a certain period of time, ManualResetEventSlim resorts to a regular event handle wait. It adds approximately 40 ns to latency.

EventBus IdleMode.Wait is the default idle mode, which is optimal and adequate for almost every task, including UHF trading. You can change it to Sleep if you don’t care about millisecond latency but don’t want to load your CPU. Alternatively, you can change it to Spin if you have a multicore CPU and require the lowest possible latency.


Top
 Profile  
 
 Post subject: Re: OpenQuant 2014
PostPosted: Sun Feb 22, 2015 3:27 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
Session Bars

OpenQuant2014 introduces a new bar type that replaces and improves daily bar concept. The Session Bar can be added to the BarFactory with

Code:
BarFactory.Add(Instrument, session1, session2);


method call, for example

Code:
BarFactory.Add(“MSFT”, new TimeSpan(10, 30, 0), new TimeSpan(16, 0, 0));


Session bars are collected daily from session open (session1) to session close (session2) and emitted accordingly, i.e. OnBarOpen is triggered at session open (actually when the first tick arrives) and OnBar at session close.


Top
 Profile  
 
PostPosted: Tue Sep 22, 2015 4:50 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6769
Strategy persistency and portfolio recovery

You can run the StrategyManager in the persistent mode to save strategy portfolio tree and execution messages during strategy run and then restore portfolio transactions and positions next time you re-start the strategy.

All you need to do to achieve this is to set StrategyManager.Persistence into one of StartegyPersistence options in the scenario.

• None – do nothing
• Load – load execution messages and restore portfolios and orders before strategy run
• Save – save portfolio tree and execution messages during strategy run
• Full – load and save

For example StrategyManager.Persistency = StartegyPersistence.Full;

Strategy portfolio is saved in the portfolios.quant file using current PortfolioServer (FilePortfolioServer by default) and orders are saved in the orders.quant file using current OrderServer (FileOrderServer by default).

If you don’t want to lose your strategy order and portfolio information in case of strategy, application or computer crash, it’s highly recommended to use (remote) FileServer for data storage.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next

All times are UTC + 3 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group