SmartQuant Discussion
http://www.smartquant.com/forums/

How to use Scenario functionality (Walk Forward analysis)
http://www.smartquant.com/forums/viewtopic.php?f=64&t=8627
Page 1 of 1

Author:  Dr. Anton Fokin [ Thu Dec 30, 2010 9:49 pm ]
Post subject:  How to use Scenario functionality (Walk Forward analysis)

Here is a simple scenario that shows how to implement brute force optimization + walk-forward backtest on out-of-sample data interval. This scenario works with SMA crossover strategy.

Code:
public class MyScenario : Scenario
{
   public override void Run()
   {
      // set in-sample data interval
      
      Solution.StartDate = new DateTime(1995, 1, 1);
      Solution.StopDate  = new DateTime(2001, 1, 1);
      
      // get reference to strategy project
      
      Project project = Solution.Projects[0];
   
      // define variables
      
      int best_length1 = 0;
      int best_length2 = 0;
      double best_objective = 0;
      
      // brute force optimization loop
      
      for (int length1 = 3;length1 <= 7;length1++)
         for (int length2 = 3;length2 <= 7;length2++)
            if (length2 > length1)
            {
               // set new parameters
               
               project.Parameters["Length1"].Value = length1;
               project.Parameters["Length2"].Value = length2;
               
               // print parameters
                     
               Console.Write("Length1 = " + length1 + " Length2 = " + length2);
      
               // start backtest
               
               Start();
               
               // calculate objective function
               
               double objective = Solution.Portfolio.GetValue();

               // print objective
               
               Console.WriteLine(" Objective = " + objective);
      
               // check best objective
               
               if (objective > best_objective)
               {
                  best_objective = objective;
                  best_length1 = length1;
                  best_length2 = length2;
               }
            }   
      
      // print best parameters
      
      Console.WriteLine("BEST PARAMETERS");
      Console.WriteLine();
      Console.WriteLine("SMA1 Length = " + best_length1);
      Console.WriteLine("SMA2 Length = " + best_length2);
      Console.WriteLine("Objective   = " + best_objective);

      // run strategy with the best parameters on out-of-sample data interval
      
      project.Parameters["Length1"].Value = best_length1;
      project.Parameters["Length2"].Value = best_length2;

      Solution.StartDate = new DateTime(2001, 1, 1);
      Solution.StopDate  = new DateTime(2005, 1, 1);      
      
      Start();
   }
}

Author:  zheka [ Fri Dec 31, 2010 8:10 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Would this feature work also in real-time? It would be great to automate trading this "walk-forward" way...

Author:  Dr. Anton Fokin [ Fri Dec 31, 2010 8:35 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Hi,

could you please describe (perhaps in a pseudo-code) what do you want to do? Something like

> switch to simulation mode
> set data interval
> optimize
> set best parameters
> switch to live mode
> start

Regards,
Anton

Author:  zheka [ Fri Dec 31, 2010 9:30 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Hi, Anton,

Yes, it is very close to what you just described.

Say, my walk-forward back-testing (optimizing on last 1000 bars - trading with the chosen parameter set on the next 200 bars) shows that a strategy is robust enough to be traded live.

And then the question is how to emulate this walk-forward changing of parameters in real trading, especially when trading on smaller time frames? The strategy is running, the last bar of the "traded" period of 200 bars is about to end, and by the start on the next bar I would like to have a new set of parameters ready. Ideally - without stopping the currently running strategy.

Cheers and Happy New Year!

E.

Author:  Dr. Anton Fokin [ Fri Dec 31, 2010 10:43 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Yes, we will definitely add the possibility to switch to live trading in the scenario code.

Actually we've just started to realise what can be done with this new scenario functionality and I think it has fantastic perspectives. We just need to be very careful adding new features to the scenario engine, otherwise you can easily burn all your money if you have too much flexibility and can develop a complex code that switches between simulated and live trading, changes parameters... especially if you run it unattended.

Though you can do it right in the strategy code anyway :wink:

Happy New Year,
Anton

Author:  douv [ Fri Dec 20, 2013 11:57 am ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Hi,
I've used this pattern to optimize my params but after each loop the value in Solution.Portfolio.GetValue() or Solution.Performance.Equity remains the same.
shouldn't the performance / drawdown series be "cleaned" automatically before when a new "Start()" session is initiated?
Is there a way i can reset them them after each loop?

Author:  Baraz Sergey [ Fri Dec 20, 2013 1:05 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Hi,

Yes, the portfolio value and performance should be cleared automatically before each optimization loop. Unless you set RestOnStart=false, is it the case?

By the way, does portfolio.GetValue() show the initial portfolio value? If yes, then most probably your system doesn't trade at all.

Regards,
Sergey.

Author:  douv [ Fri Dec 20, 2013 3:43 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

Hi,
ResetOnStart as far as i can see is by default on True. I'm in no place alter it to false.
Also i've debugged it between loops and it remained True.
I do have transactions. I've manually executed the strategy with the params from the optimize and i can clearly see difference in PnL and Drawdown. Maybe the number of positions should not change from loop to loop but the PnL and drawdown will sure do.

I've looked on the Option --> Memory Management and saw that "Remove Done Orders" is marked. Is there any relation?

Author:  douv [ Fri Dec 20, 2013 6:16 pm ]
Post subject:  Re: How to use Scenario functionality (Walk Forward analysis

I found the reason for my mistake.
it's now working as expected.
Thank you for your help.

Page 1 of 1 All times are UTC + 3 hours
Powered by phpBB® Forum Software © phpBB Group
https://www.phpbb.com/