SmartQuant Discussion

Automated Quantitative Strategy Development, SmartQuant Product Discussion and Technical Support Forums
It is currently Thu Nov 26, 2020 6:39 am

All times are UTC + 3 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
PostPosted: Wed Feb 02, 2011 6:21 am 
Offline

Joined: Sat Jan 22, 2011 1:40 am
Posts: 73
Hello,

Is there a way to check data integrity? Recently I downloaded 1 min data for AAPL and found big gaps on the chart during market hours. Look at the other my post about my backtesting problem. I have many instruments and I need to do it automatically without charts.

Thank you very much,
Paul.


Top
 Profile  
 
PostPosted: Wed Feb 02, 2011 10:58 am 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6817
You can develop a simple script (script is not a strategy, just a C# code that you can run in OpenQuant) in the Script Manager that will check all the stcosk. The next thing is to define integrity criterion and take care of holidays and weekends.


Top
 Profile  
 
PostPosted: Wed Feb 02, 2011 11:10 am 
Offline

Joined: Sat Jan 22, 2011 1:40 am
Posts: 73
Anton,

Thank you for your reply.
That is exactly what I was trying to do using the simple strategy I provided in my previous post.
I am not doing any processing, just market buy and then market sell on every other bar. I included working market hours and working days only. Just need to implement holidays, but it is not a problem.

The criteria - if OpenQuant can process these data then it is OK. If I would implement my own criteria and OpenQuant does not take it then it will not help me at all.

Next step - how to fix the problem. I have no idea.

Regards,
Paul.


Top
 Profile  
 
PostPosted: Wed Feb 02, 2011 11:15 am 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6817
But in any case it's a good idea to visually inspect your data, unless you purchase the data from a trusted source (and yet I would spend some time on the basic visual data inspection). Companies like TickData hire tens of employees in China to manually check the data that they sell. Nothing works better than a human eye. Don't forget that you should also check the data for outliers, spikes, bad ticks, etc.


Top
 Profile  
 
PostPosted: Wed Feb 02, 2011 11:19 am 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6817
> The criteria - if OpenQuant can process these data then it is OK. If I would implement my own criteria and OpenQuant does not take it then it will not help me at all.

I am not sure I follow. OpenQuant can process any data. It's your responsibility to prepare a good quality data for backtesting. OpenQuant provides access to historical database, so that you can read, delete and add data entries.


Top
 Profile  
 
PostPosted: Wed Feb 02, 2011 12:02 pm 
Offline

Joined: Sat Jan 22, 2011 1:40 am
Posts: 73
Anton,

OpenQuant can not process missing or corrupted data. Plugin is responsible for populating the data. If the data has gaps ( and for AAPL no gaps!) then something just does not work properly.
Using my strategy I just can take a look how many trades I have to check if I have gaps. I can do it in OpenQuant and AmiBroker and compare results. If results the same then the problem with the data if not then the problem with plugins. The source of the data is the same - IQFeed. Strategy is extremely simple, no bugs here. I am considering to use OpenQuant for a very big financial project so I am trying to test everything.

Regards,
Paul


Top
 Profile  
 
PostPosted: Wed Feb 02, 2011 12:08 pm 
Offline

Joined: Tue Aug 05, 2003 3:43 pm
Posts: 6817
Hi Paul,

since IQFeed adapter is developed by a third party, we don't really know what's going on there. There were some problems with historical data download reported by other users recently. I know you are in contact with IQFeed adapter developers, so please talk to them to see that you understand all the limitation of the current version of their adapter.

Regards,
Anton


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC + 3 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group