SmartQuant Discussion

Automated Quantitative Strategy Development, SmartQuant Product Discussion and Technical Support Forums
It is currently Fri Sep 25, 2020 6:34 pm

All times are UTC + 3 hours




Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Thu Oct 28, 2010 1:13 am 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
This plugin provides full access to the financial datastream from IQfeed.

You can DOWNLOAD the FULL .DLL from:

http://www.microchipc.com/wiki/openquant_iqfeed/NFIQFeedProvider_v1-00_forum.rar
(link updated 2011-01-25).

This version is a complete rewrite of the original IQfeed driver v0.95, and an update from v0.99 released in late 2010.

Here is how to install the plugin:

Image

Functionality:

1. Supports real time level 1 (bid/ask) market data.
2. 10+ years of daily history (O,H,L,C,V,OI).
3. Several years of 1-Minute history (Forex back to Feb 2005, Eminis back to Sept. 2005, Stock/Futures/Indexes back to May 2007).
4. Has an option to subscribe to the option chains associated with an underlying.
5. Auto login - just enter your username/password in the properties box.
6. Tested for the last 2 months in a production system.

Notes:

I've found that IQfeed is pretty much rock solid; it never seems to drop out (unlike other providers that I won't mention here).

How to install the plugin:

1. Download the release .rar file (above).
2. Follow the instructions in the "readme.txt" file.

How to download historical data using the plugin:

Image

Image

Image

Image


Last edited by smt52 on Tue Jan 25, 2011 11:23 pm, edited 6 times in total.

Top
 Profile  
 
PostPosted: Thu Oct 28, 2010 1:23 am 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
Here is the contents of the "readme.txt" file, which is contained in the release you can download above:
---start readme.txt--
Here is the contents of the "readme.txt" file, which is contained in the release you can download above:
---start readme.txt--
* IQfeed driver for OpenQuant 2.
* Release v1.00
* www.microchipc.com
* support[at]microchipc.com
* 2011-01-25

This archive contains the completed OpenQuant IQFeed data provider solution
(including options chains synchronization).

Features:

* Compatible with OpenQuant v2.9.4 and upwards. Tested with OpenQuant v3.0.3.
* Driver for IQfeed v4.7.2.0 (latest as of 2010-10-27).
* Supports forex, stocks, and options.
* Use OpenQuant as a Data Manager to automatically record all ticks and trades for any combination of forex/stocks/underlying/options symbols.
* Automatic login for IQfeed (specify username and password in properties).
* Automatic reconnect if IQfeed drops the connection at any stage.
* Reading of option chains associated with an underlying from the IQfeed driver.
* It is possible to automatically subscribe to the correct option chains for an underlying.

Instructions:

* 1. Go into "\copy to OpenQuant-bin\version without logging\.
* 2. Copy all files into "%Program Files%\SmartQuant Ltd\OpenQuant 2\Bin".
* 3. Start OpenQuant. Select "Tools >> Options >> Providers >> User" and then browse to the .dll "NFIQFeedProvider.dll".
* 4. Select Providers tab, select "NFIQFeed", right click, "Properties".
* 5. Fill in your IQfeed username under "IQFeedUser".
* 6. Fill in your IQfeed password under "IQFeedPassword".
* 7. Set provider connect timeout to 60 sec.
* 8. Right click on "NFIQfeed", and click "Connect".
* 9. Wait about 30 seconds. Do not touch the keyboard - although the IQfeed login box will pop up temporarily, it will log in automatically using the username/password you specified in the properties tab.

Folder "OpenQuant":

This folder contains OQ strategy code, which performs options synchronization inside OpenQuant. To install:

* 1. Create new solution in OQ.
* 2. Add attached project to this solution.
* 3. Select Solution | Manage References and add reference to NFIQFeedProvider.dll previously saved into the "%Program Files%\SmartQuant Ltd\OpenQuant 2\Bin".
* 4. Compile solution in OQ.
* 5. Select NFIQFeed as default live provider in "Tools >> Options >> Modes".
* 6. Run strategy and wait for IQFeed to connect and then for at least 1 minute.
* 7. Strategy will report process into the Output window. Options for all existing instruments in the instrument tree will be added automatically.
* 8. Options will be synchronized again in a hour (this is changable).

To use OpenQuant as a Datamanager to record all ticks/trades for any underlying + associated options:

We will use AAPL as an example.

* 1. In OpenQuant, go into "Tools..Options..Configuration..General", and set "Bar array length", "Trade Array Length", and "Quote Array Length" from -1 to 1000. If you fail to do this, the system will eat up memory continuously, until it crashes some days later when it hits its memory limit which could be many gigabytes.
* 2. Subscribe to an underlying, i.e. AAPL.
* 3. Run the strategy (above) to auto-subscribe to all of the option symbols associated with AAPL. You should see all the option chains associated with AAPL appear in the list of subscribed option symbols to the left.
* 4. In OQ, select "Data..Import..Realtime..NFIQfeed".
* 5. Tick the boxes for the data to record.
* 6. Press the play button, and you will see that OQ will start to record ticks + trades for both AAPL and all of the associated options.

Compiling:
* Requires Microsoft Visual Studio 2010. Note that the project is NOT compatible with Visual Studio 2008 without unspecified modifications.
* Always compile in "Release" mode. If you compile in "Debug" mode, it adds the EX_DEBUG flag in compile options which is designed to make it work with the console wrapper. The console wraper is designed so to verify that memory usage for the program is stable.

FAQ:

Q. Whats the difference between "\copy to OpenQuant-bin\version with logging" and "\copy to OpenQuant-bin\version without logging"?
A. The version with logging generates log files. You would never need this unless you are altering the driver. The verison has been compiled with the "SILOG" option switched on. See "Properties" for "NFIQFeedProvider", then "Build", and under "Conditional Compilation Options" type "SILOG". To view the logs that the driver generates during operation, you will have to purchase SmartInspect. See the file "SmartInspect log extract.txt" for an example of the log.

Q. Can I download historical data?
A. Yes. Select "Data..Import..Historical Data..NFIQFeed", then import the data you want.

Q. What is "AutoConnect" for in the properties for "NFIQfeed"?
A. If this property is set to "true", once the driver is connected, try killing the "iqconnect.exe" process from task manager. Wait 30 seconds, IQfeed will automatically be reloaded and the driver will start again; no user interaction is required. This feature ensures that no matter what happens, IQfeed will always be available to supply OQ with a datafeed.

Versions:

v1.00. Release 2011-01-25. Fixed memory leak issue. Fixed !NO_DATA! issue.
v0.99. Released late 2010. Complete rewrite of v0.95.
v0.95. Released 2008.

Errata:

* Errata 1: RAM usage of OQ will continuously increase slowly until it reaches the system limit unless you select "Tools..Options..Configuration..General", and
set "Bar array length", "Trade Array Length", and "Quote Array Length" from -1 to a fixed number. This is the number of trades to keep, per symbol. A value of around
2000 should be about right. SEe viewtopic.php?p=24754#24754
* Errata 2: Cannot load 1-min data more than 60 days into the past. This could be fixed with a driver update.

Source code:

I am officially prevented from publicly disclosing the source code by the terms of the IQfeed agreement, but if you email me we can work on the source code together to implement bugfixes, etc.

Help:
* Email Shane Tolmie, support[at]microchipc.com.
---end readme.txt---
---end readme.txt---


Last edited by smt52 on Tue Jan 25, 2011 11:21 pm, edited 1 time in total.

Top
 Profile  
 
PostPosted: Wed Nov 24, 2010 7:02 pm 
Offline

Joined: Wed Mar 10, 2010 1:42 am
Posts: 63
Thanks for posting the update Shane. What significant changes are there from the previous version? I know you state that it was completely rewritten, is it more stable? faster?


Top
 Profile  
 
PostPosted: Mon Nov 29, 2010 2:27 am 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
scrubolio wrote:
Thanks for posting the update Shane. What significant changes are there from the previous version? I know you state that it was completely rewritten, is it more stable? faster?


It should be more stable. The code architecture is now a lot cleaner. It was rewritten from the ground up using lessons from the last iteration.


Top
 Profile  
 
PostPosted: Fri Dec 31, 2010 6:25 pm 
Offline

Joined: Wed Feb 27, 2008 7:37 am
Posts: 28
I just downloaded this plug in and tried on OQ3.0.2 and have no luck getting it to even appear under the Market Data tree... Is this compatible with OQ 3.x?

Thanx


Top
 Profile  
 
PostPosted: Fri Dec 31, 2010 6:53 pm 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
csharp_student wrote:
I just downloaded this plug in and tried on OQ3.0.2 and have no luck getting it to even appear under the Market Data tree... Is this compatible with OQ 3.x?

Thanx


OQ v3 was released 12 days ago. we have not had a chance to test it yet. It all depends on whether OQ is prepared to give us a copy of OQ so we can maintain the plugin (feel free to lobby them!).


Top
 Profile  
 
PostPosted: Sun Jan 16, 2011 5:15 pm 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
smt52 wrote:
csharp_student wrote:
I just downloaded this plug in and tried on OQ3.0.2 and have no luck getting it to even appear under the Market Data tree... Is this compatible with OQ 3.x?

Thanx


OQ v3 was released 12 days ago. we have not had a chance to test it yet. It all depends on whether OQ is prepared to give us a copy of OQ so we can maintain the plugin (feel free to lobby them!).


Just tested it, and it works perfectly on OQ v3. If you had no luck getting it to appear under the Market Data Tree, follow the instructions, and look for the "NFIQfeed" driver under the Market Data Tree. You'll have to copy "NFIQFeed.dll" to \bin, and go into options to add the user provider (the readme.txt gives far more details).


Top
 Profile  
 
PostPosted: Thu Jan 20, 2011 12:56 pm 
Offline

Joined: Sun Nov 21, 2010 5:46 pm
Posts: 51
Plugin doesnt works correctly if more than 2 month of 1-minutes data requested. It gives error:

Attachment:
1-18-2011 12-49-32 AM.png
1-18-2011 12-49-32 AM.png [ 32.29 KiB | Viewed 13817 times ]


IQfeed account is OK cause IQfeed plugin for amibroker works perfect and downloads 1-minutes data for year range.

Problem occurring with all symbols. Plugin installed with all instructions of readme.
OpenQuant version 2.9.13


Top
 Profile  
 
PostPosted: Thu Jan 20, 2011 1:29 pm 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
Fabio wrote:
Plugin doesnt works correctly if more than 2 month of 1-minutes data requested. It gives error:

Attachment:
1-18-2011 12-49-32 AM.png


IQfeed account is OK cause IQfeed plugin for amibroker works perfect and downloads 1-minutes data for year range.

Problem occurring with all symbols. Plugin installed with all instructions of readme.
OpenQuant version 2.9.13


I reproduced the problem, and you are correct: it doesn't work on 1-min data. The reason is that the plugin is currently designed for tick data (both historical and realtime), and there is only 3 months of tick data available from IQfeed via the standard API calls to retrieve tick data. If it is downloading 1-min data, it is downloading tick and converting it into 1-min.

For a workaround, if you want 1-min data, you can always download the 1-min .csv files using QCollector or AmiBroker, then import these into the OpenQuant database. The latest version of OpenQuant

This problem would require a fix at the driver level to make a different API call. Contact me if you want the opportunity to work on the driver source code.


Last edited by smt52 on Thu Jan 20, 2011 1:45 pm, edited 1 time in total.

Top
 Profile  
 
PostPosted: Thu Jan 20, 2011 1:44 pm 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
smt52 wrote:
When I run the driver, it always blanks out the username/password boxes - how do I insert my username/password without it getting erased?

Right click on the driver, go into "Properties", and change the username/password boxes:

Image

This username/password is passed directly to the IQfeed driver when it logs in. This has the advantage that IQfeed can log in automatically again if it loses the connection, and its one small step towards making OpenQuant more automated (you don't have to worry that IQfeed will lose its password settings if its reinstalled).

This might create a conflict if multiple programs use IQfeed (i.e. QCollector and OpenQuant on the same machine). It's very possible to add a new property to the driver which disables this autologin feature, contact me if you want the opportunity to work on the source for the driver.


Top
 Profile  
 
PostPosted: Thu Jan 20, 2011 7:01 pm 
Offline

Joined: Sun Nov 21, 2010 5:46 pm
Posts: 51
smt52 wrote:
Fabio wrote:
Plugin doesnt works correctly if more than 2 month of 1-minutes data requested. It gives error:

I reproduced the problem, and you are correct: it doesn't work on 1-min data. The reason is that the plugin is currently designed for tick data (both historical and realtime), and there is only 3 months of tick data available from IQfeed via the standard API calls to retrieve tick data. If it is downloading 1-min data, it is downloading tick and converting it into 1-min.

For a workaround, if you want 1-min data, you can always download the 1-min .csv files using QCollector or AmiBroker, then import these into the OpenQuant database. The latest version of OpenQuant

This problem would require a fix at the driver level to make a different API call. Contact me if you want the opportunity to work on the driver source code.


Thank you, im already using same way. 1. Download 1min data with ami. 2. Expo to csv. 3. Delete 1 column in file with excel. 4. Impo csv to OQ.
Im testing pare trading script, and i need to check hundreds of securities. So, this is not the best way to download all of them in OQ :(

Any chances you will fix the "tick" issue with your plugin?


Top
 Profile  
 
PostPosted: Thu Jan 20, 2011 7:40 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6817
Hi,

perhaps you can write a simple script that will read csv file and import data in OQ? You can process many instruments / files at once this way.

I think we have an example of such a script either in OQ (demo scripts) or somewhere on the forum.

Regards,
Anton


Top
 Profile  
 
PostPosted: Thu Jan 20, 2011 8:46 pm 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
Fabio wrote:
Thank you, im already using same way. 1. Download 1min data with ami. 2. Expo to csv. 3. Delete 1 column in file with excel. 4. Impo csv to OQ.
Im testing pare trading script, and i need to check hundreds of securities. So, this is not the best way to download all of them in OQ :(

Any chances you will fix the "tick" issue with your plugin?


To be honest, most of our backtesting is in Matlab, we don't use OQ for actual backtesting, so there is no incentive for us. The IQfeed driver works well on live market data. You're welcome to alter the source code to fix this issue, however, it would require considerable knowledge of C#.


Top
 Profile  
 
PostPosted: Thu Apr 07, 2011 6:01 am 
Offline

Joined: Fri Apr 02, 2010 10:08 am
Posts: 44
Hi,


I was wondering if anyone is able to get the feed for TICK.Z and VIX.XO? I can get the quotes for the stocks without a problem but cannot get the feed for these indexes.

thanks

geoarb


Top
 Profile  
 
PostPosted: Fri Apr 08, 2011 10:27 pm 
Offline

Joined: Sun Nov 16, 2008 2:55 pm
Posts: 50
geoarb wrote:
Hi,


I was wondering if anyone is able to get the feed for TICK.Z and VIX.XO? I can get the quotes for the stocks without a problem but cannot get the feed for these indexes.

thanks

geoarb


I've replied to you offline.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC + 3 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group