How Our Backtesting
Pipeline Works

1. Overview — How We Test Strategies

Capo Horn Lab provides rigorous, institutional-grade backtesting for systematic trading strategies. Unlike retail backtesting platforms that rely on OHLC bar data and simplistic execution models, we use tick-level market data, realistic slippage models, and statistical validation techniques to give you an honest assessment of whether your strategy has a genuine edge.

Our process is: you describe your strategy rules, we encode them in our pipeline, run the analysis against historical tick data, and return a comprehensive report. We never sell strategies or signals. Our only product is rigorous analysis.

2. The Backtesting Pipeline

Every strategy submitted to Capo Horn Lab goes through the following pipeline:

Step 1 — Strategy Encoding

Our quantitative team translates your strategy description into executable code. Entry and exit rules, stop losses, take profit levels, position sizing, and any filters or conditions are encoded exactly as specified. We document every assumption and parameter choice so there is no ambiguity.

Step 2 — Data Sourcing

All historical data is sourced from Databento, providing tick-level market data for futures contracts including ES, NQ, CL, 6E, and YM. Data includes bid/ask quotes, trade prices, volumes, and timestamps at nanosecond precision. We handle roll adjustments, contract expiries, and corporate actions automatically.

Step 3 — Simulation Engine

The backtest engine processes tick by tick, evaluating strategy conditions at each market event. Order execution uses a realistic model calibrated to market conditions at the time of execution. Configurable parameters include:

Parameter	Description	Default
Slippage	Basis point cost per side, configurable per request	0.5 bps
Commission	Per-contract fee per side	$2.50 per side
Execution model	Bar-close, limit, or market-at-tick	Bar-close + delay
Order delay	Simulated latency between signal and fill	1 tick

Step 4 — Statistical Validation

The raw equity curve goes through multiple validation layers:

• IS/OOS Split: The data is split into in-sample (development) and out-of-sample (validation) periods. The strategy is tested blind on the OOS data.
• Monte Carlo Simulation: Thousands of shuffled trade sequences simulate alternative paths the strategy could have taken, producing confidence intervals around performance metrics.
• Parameter Stability: If your strategy has parameters, we test sensitivity across a grid of values to check for overfitting.
• Distributional Analysis: Trade returns are tested for normality, autocorrelation, and regime dependency.

Step 5 — Report Generation

All results are compiled into a structured report (see Section 4) with full methodology documentation, raw trade logs, and a clear verdict.

3. Key Metrics Explained

Your report includes a range of performance and risk metrics. Here is what each one means:

4. How to Read Your Report

Each backtest report is structured into clear sections. Here is what to look for in each:

Executive Summary

The first page gives you the bottom line: a verdict box that says Bullish, Neutral, or Bearish on the strategy. This is not a trading recommendation — it is a summary of the statistical evidence for or against the strategy having an edge.

• Bullish: Passes all validation tests with statistically significant results.
• Neutral: Shows some positive signals but does not meet the threshold for statistical significance across all tests.
• Bearish: Fails one or more validation tests. The strategy likely has no edge, or the data actively contradicts the hypothesis.

Equity Curve

A chart of cumulative P&L over the test period. Look for: steady upward slope (good), flat periods with sharp drops (suspect), equity curve that goes up and never down (almost certainly overfitted or data-snooped). We overlay Monte Carlo confidence bands to show the range of plausible outcomes.

Trade Log

Every individual trade with entry/exit timestamps, prices, P&L, and R multiples. Review this for: clustered losses (regime dependency), unrealistic fills (price improvement beyond market conditions), and pattern of winners vs losers.

Distribution Charts

Histograms of trade returns, R multiples, and holding periods. A strategy with genuine edge typically shows a right-skewed distribution (more small wins than small losses, with occasional large wins). A symmetric or left-skewed distribution is a warning sign.

Validation Results

Detailed results of IS/OOS tests, Monte Carlo simulations, parameter stability checks, and any other validation procedures. This section is where we separate genuine edge from overfitted noise.

5. Interpreting Results

When a Strategy Passes

A Bullish verdict means the strategy survived all validation checks:

• Sharpe ratio exceeds 1.0 with the lower bound of the 95% confidence interval above 0.5
• Monte Carlo P-value < 0.05
• IS/OOS performance gap < 30% (the strategy does not fall apart on unseen data)
• Parameter stability — the strategy is not overly sensitive to small parameter changes
• Trade count sufficient for statistical significance (typically > 30 trades)

Caveat: Passing these tests does not guarantee future performance. Markets change. What worked in 2020 may not work in 2025. The results are a data point, not a prophecy.

When a Strategy Fails

A Bearish verdict is valuable information. Common reasons:

• Insufficient edge: The strategy barely beats random, or the Monte Carlo simulation shows the results are within the noise band.
• Overfitting: The strategy looks great on in-sample data but collapses out-of-sample. Common symptoms: too many parameters, tight stops that look calibrated to past volatility.
• Data snooping: The strategy rules were derived from looking at the same data they are tested on. Our IS/OOS split catches this.
• Market regime shift: The strategy worked in some periods but failed in others. We flag this with regime-dependent analysis.

The Neutral Verdict

A Neutral verdict is the most common outcome for strategies that are not obviously bad but also not convincingly good. It means: the data shows some positive signals, but not enough to cross the statistical significance threshold. This is not a failure — it is an invitation to refine the hypothesis and test again.

6. Understanding Limitations

Every backtesting methodology has limitations. Being transparent about them is part of our commitment to honest research:

• No forward-testing substitute: A backtest is historical analysis. It cannot predict how a strategy will behave in market conditions that have no historical precedent.
• Execution assumptions: Our slippage and fill models are realistic but cannot capture every edge case — particularly in fast market conditions or during news events.
• Survivorship bias: We use current contract specifications. Delisted contracts or rules changes that would have affected the strategy are noted but cannot always be fully reconstructed.
• Regime dependency: A strategy that works across multiple market regimes (trending, ranging, high vol, low vol) is more robust than one that only works in one. Our reports flag regime dependency when detected.
• Psychological factor: No backtest captures the emotional difficulty of executing a strategy through a drawdown. A strategy that backtests at a Sharpe of 1.5 may still be impossible to follow in real trading.

7. Common Questions

How much data do you need to test a strategy?

We recommend at least 2 years of data for statistically meaningful results. More data means more reliable conclusions. The exact amount depends on the strategy's trade frequency — a daily strategy needs more calendar years than a 5-minute strategy.

Can you test strategies with multiple instruments?

Yes. Multi-instrument and portfolio-level backtesting is supported. We can test strategies that trade different instruments simultaneously, with correlation analysis and portfolio risk metrics included in the report.

How do you handle contract rollovers?

Futures contracts expire. We handle rolls automatically using a configurable roll schedule and method (last day, volume-based, or custom date). The report documents the roll methodology used and any adjustments applied.

What happens if I submit a strategy with ambiguous rules?

We will contact you for clarification before running the test. All assumptions are documented in the report. If we cannot resolve the ambiguity, we will flag it and suggest alternatives.

Can I get the raw data from my backtest?

Absolutely. Every report includes a full trade log with entry/exit prices, timestamps, P&L per trade, and equity curve data. We believe in reproducible research — you should be able to verify every number in your report.