TL;DR
Publishing only a favorable window can hide the same strategy’s weaker history. The five-dividend-grower simulation returned 44.80% versus stored SPY’s 33.99% from January 2025 through October 2026, but 340.90% versus 419.23% from January 2014 to the same endpoint. Durations, exposure and commissions differ; this is a selection example rather than proof of overfitting. Retain every candidate, losing result and operational failure, then separate earlier selection from a frozen later test.
Compare two completed backtests
- Strategy return (%) 2026-10-07
- 44.8
- SPY baseline return (%) 2026-10-07
- 33.99
The recent period looks better
From January 2, 2025 through October 7, 2026, the candidate returned 44.80% and the stored SPY baseline returned 33.99%. The teal curve tells an appealing story on its own.
Read this period with its dates and costs. A favorable historical window cannot establish how the strategy will perform next.
- Strategy return (%) 2026-10-07
- 340.9
- SPY baseline return (%) 2026-10-07
- 419.23
The longer period changes the story
Apply the same strategy rules from January 2, 2014 through October 7, 2026. The candidate returned 340.90%, while the stored SPY baseline returned 419.23%. Publishing only the recent period would hide this result.
Keep both results in the research record. Different durations and exposures limit the comparison; this is a selection example, not proof of overfitting.
Compare both periods in Aurora
Open the prepared request in Aurora. It includes the example’s actual inputs, so you can ask about the part you want to check.
Review the explanation against the evidence on this page. Account changes and research jobs require a separate decision.
The same dividend rules tell two different stories
Show the table · 2 rows, 6 columns
Scroll sideways to compare all columns. Select a heading to sort.
| 2025-01-02 | 44.80% | 33.99% | 24.33% | 49 | $24.15 |
| 2014-01-02 | 340.90% | 419.23% | 24.16% | 422 | $331.98 |
No matching rows. Clear the filter to see all records.
The same five-dividend-grower rules beat the stored SPY return in the 2025 window and lagged it in the longer 2014 window. Publishing only the recent curve would hide that difference. These inspected periods are historical examples, not an untouched final test or proof of overfitting.
Each run starts with $10,000 and reinvests dividends. The strategy pays 0.1% stock transaction fees; the stored SPY baseline pays no candidate commissions. Historical point-in-time SP500 membership is unverified. Different assets, costs and exposure limit the comparison.
Audit the population behind the published corpus
Read the archive population and missing history
The available archive contains 222,010 recorded runs, with its newest record dated 2026-09-06. The October reanalysis selects a daily stock cohort without options, excludes optimizer-labeled runs and keeps finite-Sharpe records lasting 365 through 2,000 days.
The selected records do not cover every attempted strategy, and the archive does not report how many jobs failed. Author origin is unknown. The archive does not identify which optimizer candidates were chosen before a later test, so it cannot establish optimizer superiority or clean unseen performance.
Scroll sideways to compare all columns. Select a heading to sort.
| Available source runs | 222010 | The retained archive, not the live database |
| Optimizer-labeled source runs | 74518 | Recorded label across the source; excluded from this ranking cohort |
| Eligible daily stock runs | 102781 | After asset, cadence, duration, origin-label and metric filters |
| Incomplete or unsupported records excluded | 12233 | Typed normalization rejects missing or unsupported execution inputs |
| Repeated execution records collapsed | 47698 | One representative per normalized rule/action and recorded execution contract |
| Distinct available recorded execution contracts | 42850 | Normalized typed rules/actions and recorded execution inputs |
| Zero-return contracts retained | 3013 | Zero return is not proof of zero fills |
No matching rows. Clear the filter to see all records.
Audit the contest before copying a finalist
Read audit the contest before copying a finalist
Open the actual completed study in Optimization and inspect every available candidate and fold, not just the first card. Save the original registry and the complete full-field portfolios. Separate candidates that completed and lost from jobs that failed operationally. If the tool output contains only a truncated leaderboard, retain the truncation and do not call it the full trial set.
For each finalist, record which validation window selected it and which outer windows were used to evaluate it. Then record any cross-fold summary used to choose the final assembled portfolio. That final choice needs its own untouched test because it used earlier outcomes.
The winner needs the size of the contest
Read the winner needs the size of the contest
A result selected because it was the strongest among many trials is harder to interpret than a predeclared single-rule test. The selection can happen through an optimizer, an agent conversation, manual parameter changes, asset screening or which chart gets published.
Record every evaluated candidate, including failed runs that produced usable evidence. A new prompt, random seed or renamed study does not erase the other attempts. Operationally failed jobs should be identified separately from strategies that completed and lost.
Separate the fold choice from the hindsight summary
Read separate the fold choice from the hindsight summary
NexusTrade’s walk-forward runner selects a fold candidate from validation statistics before promoting that candidate into the later outer test. A cross-fold retrospective summary may inspect all completed outer windows. Using that summary to pick a final book introduces another selection decision.
Do not report the retrospectively strongest candidate as though that exact choice had been made before the first test window. Freeze the assembled portfolio and evaluate it in a separate final holdout. If no candidate satisfies configured floors, show the fallback warning rather than certifying the best of the failures.
Publish the selection ledger
Read publish the selection ledger
Save this ledger before publishing a winner. If earlier trials are unavailable, call the search history incomplete. A polished table of finalists cannot establish a complete search history.
candidate | first declared | trial count | selection window | test window | failed floors | included in report
[all candidates, including losses and rejected variants]
final choice | exact rules hash | information used | independent final holdoutCheck who disappeared from the universe
Read check who disappeared from the universe
A current list of successful companies can exclude delisted or failed names that were eligible at the historical start. An eligible-as-of universe needs a defined membership source and dates. A backtest over a current survivor list does not establish protection against survivorship bias.
Keep asset exclusions, missing-data policies and delisting handling explicit. Excluding a name because it lacked necessary observations is different from excluding it because its completed return was poor; both need an auditable reason.
No comments yet.