Group Strategy Tester

Group Strategy Tester (former Strategy Variance Test) tool lets you run backtests on multiple strategies, symbols, and time frames. It runs multiple backtests based on the options you choose and then presents the results in one spreadsheet. This tool is built for the purpose of exploring variance of your backtest metrics in different circumstances. One can pursue different goals, but analyzing variance is still the key here.

Here are a few examples of how you can use that:

1. Test strategies on watch lists. That is, backtest one or more strategies on different symbols (on one or more time frames). This helps you identify which market your strategy works best for. At the same time, if you believe that these markets are fundamentally similar, then you should also explore variance of metrics. Just like with p.2 below, extreme variance could indicate that your strategy is too fragile.
2. Test one strategy on different time frames for the same market. This helps you see which time frames look more promising. Also, you can check how metrics of the same strategy change depending on symbols or time frames. Extreme variance in important metrics like drawdown or win rate could indicate that your strategy is too fragile.
3. Test different strategies on the same chart, symbol, and time frame. This helps you determine which strategy is the best fit for a given chart.
4. Test any combination of the above. Test 3 strategies on 5 symbols on 2 time frames and see what happens.

The Widget

The Group Strategy Tester widget has four main parts.

1. Configuration Panel: This is where you set up your variance test. You choose strategies, time frames, backtest depth and symbols you want to test at. You can load a watch list or a scanner as a source for the list of symbols.
2. Variant Table: This is a spreadsheet displaying all the tabular metrics for each backtest from the list of your variants defined above. The grid helps you assess which variants excel in different aspects.
3. Charts: The third part of this widget features a set of customizable charts. You have the flexibility to customize which metrics represent the horizontal axis, vertical axis, and bubble size. While default metrics are set, you can easily change them to suit your analysis.

Variant Table

This table is a straightforward spreadsheet. Each row represents one variant. Each column represents a metric, colored as a heat map. The colors range from green (the best value in a column) to transparent (the worst one). For example, in the R/R column, the highest RR is green, and the lowest is transparent. For Drawdown column, the highest drawdown is transparent and the smallest is green. You can sort and filter columns, as well as choose which metrics do you want to see. Entire row being green is a rare case; most times, specific variants are good in specific areas, requiring your judgment.

Charts

You can visualize the results on multiple customizable charts. Each chart can be a bubble chart (illustrating up to 3 metrics) or a column chart (illustrating one metric). The default configuration of charts is explained below, but you can customize the charts as you please.

On bubble charts, bubbles are color-coded on a gradient from orange to green. The best combinations of X and Y metrics show as green bubbles, the worst as orange, and the rest with colors in between

Chart 1: Reward/Risk vs Win% vs Positions (Bubble chart)

The first chart displays Risk-Reward (R/R) against Win Percent. If your bubbles land in the red area, it suggests that with the given R/R and Win Percent, your strategy will eventually hit zero (if you assume that RR and Win Percent persist over time). Red area is a no-go zone on this chart, and it's identical to a corresponding no-go flag in our Tabular Data.

Bubble size on this chart is determined by the amount of positions in the corresponding backtest. We assume that more positions means better statistical significance for the overall backtest.

On this chart, the best bubbles will be big bubbles (a lot of positions) at the top (high Win%) right (high R/R) corner of a chart. Pretty often, you'll see your bubbles sitting at the edge of the red area, which indicates that a given strategy has not managed to extract any alpha at a given market and was simply breaking even.

On the chart above, the vast majority of bubbles are in the red area, which means that they were not viable mathematically: strategies with the combination of R/R and Win% like that will go to zero over time. Strategy 1 at USDCAD,240 and Strategy 2 at USDCAD,M variants are above the red area, with Strategy 1 at USDCAD,240 having more positions than Strategy 2 at USDCAD,M. If used in a straightforward way, this chart implies that Strategy 1 at USDCAD,240 was the best option in terms of R/R and Win%.

Chart 2: Avg.Return vs Return St.Dev vs R/R (Bubble chart)

The second chart plots average return against standard deviation of returns. It helps identify which variant offers the best average return with the least deviation. Green bubbles indicate high average returns with minimal fluctuation. This chart is handy for assessing if returns are evenly distributed or if there are significant outliers, giving you insights into the consistency of your strategy.

On the chart above, the best average returns were generated by Strategy 1 at EURUSD,W. Strategy 2 at EURUSD,M has the most uneven distribution of returns and average return below zero (which is a no go). Strategy 2 at USDCAD,M was the best option in terms of Avg.Return and Return St.dev.

Chart 3: Exposure (Column chart)

The third chart illustrates time spent in market (exposure) as a column chart.

Limitations

Everyone can run group tests with up to 53 combinations. For example, if you select three strategies and 10 symbols on 1 time frame, the total variants computed will be 30 (3 * 10 * 1). Depending on your plan, you can have your limit raised up to 500 combinations. Please contact the support for custom configurations.