Explore more topics
Published model evaluations

Eleven models building an options strategy

Published 2026-04-06: 11 model runs; same task, account context, tools and 25-iteration budget. Goal-oriented evaluator score / 100. See the results, how they were measured and where the findings stop.

As of 2026-04-06

Jump to a section

TL;DR

Gemini 3 Flash Preview led the April 2026 eleven-model options test with 66 out of 100 and a mixed verdict after 45.5 minutes. Its selected strategy's reported 2022, 2023 and 2024 returns averaged 15.66%, and the evaluator recommended more iteration. GPT-5.4 stopped after a plan and scored 0. Each model had one run with the same task and 25-iteration budget; the study did not establish achieved account doubling or a general model ranking.

Published
2026-04-06
Published model rows
11
Evidence
Historical task evaluation

Published results

Overall scores in the shared options task

Scale: 0 to 66 / 100 · zero baseline

April 6, 2026 task

  • Flash Preview+66 / 100
  • Flash Lite+53 / 100
  • Sonnet 4.6+50 / 100
  • Pro Preview+50 / 100
  • Gemma 4 31B+50 / 100
  • MiMo v2 Pro+47 / 100
  • Kimi K2.5+5 / 100
  • Grok 4.20+4 / 100
  • GPT-5.4-mini+1 / 100
  • GPT-5.40 / 100
  • Mistral Small0 / 100

One run per model on April 6, 2026, with the same task and 25-iteration budget. The leading verdict was mixed and recommended further iteration. These scores are not account returns.

Show the table · 11 rows, 6 columns

Scroll sideways to compare all columns. Select a heading to sort.

Gemini 3 Flash Preview66mixed3645.5m
Gemini 3.1 Flash Lite53weak065.3m
Claude Sonnet 4.650weak2532m
Gemini 3 Pro Preview50weak2428m
Gemma 4 31B50weak03~40m
MiMo v2 Pro47weak0418m
Kimi K2.55fail0820.4m
Grok 4.204fail21110.3m
GPT-5.4-mini1fail005.3m
GPT-5.40fail0010.7m
Mistral Small 26030fail005.4m

11 model runs; same task, account context, tools and 25-iteration budget. Published 2026-04-06. Values retain their original units and grading scale.

What the result helps you choose

Gemini 3 Flash Preview led with 66/100 and a mixed verdict after 45.5 minutes. The evaluator still recommended more iteration. A fast failed run and a completed satisfactory strategy are different outcomes.

Develop an options strategy: follow the published task

A score measures how well the workflow addressed that objective. Portfolio and subagent counts describe activity; the scorecard and narrative disagree on some counts.

Step 1 of 3

The same prompt went to all 11 models

Look at my watchlist and current market conditions, design a profitable options trading strategy that I should use at Monday at open. My goal is to double my $25,000 Public Portfolio live-trading account this year.

What you should see

The models had the same watchlist, market data, account context, tools and 25-iteration budget.

Step 2 of 3

Flash acted; GPT-5.4 handed the next step back

The published Flash trace fetched market data, launched two specialist agents, read their backtests and called deployPortfolio. GPT-5.4 fetched market data and planned three tracks, then stopped with: "Please proceed by launching those subagents."

What you should see

The trace demonstrates the difference between choosing the next action and carrying it out. Its Flash process counts disagree with the preserved scorecard.

Step 3 of 3

Read the actual Flash evaluator result

The evaluator returned overallScore 66, verdict mixed and deploymentVerdict iterate_first for Regime-Adaptive Options (NVDA/META). Its listed yearly returns were 2022 +31.22%, 2023 +15.41% and 2024 +0.34%, averaging +15.66%.

What you should see

The evaluator capped strategy fitness at 3/10 because the reported average was below 30%. It recommended more iteration despite the trace showing a deployment action. Those separate test-year returns do not record achieved account doubling.

Task and tested input

Develop an options strategy for a $25,000 account with the stated goal of doubling it within a year.

  1. Published prompt excerpt

    Look at my watchlist and current market conditions, design a profitable options trading strategy

Calculate the decision from the measured rows

Scale: 0 to 66 · zero baseline

Score

  • Gemini 3 Flash Preview66
  • Claude Sonnet 4.650

Scale: 0 to 3 · zero baseline

Subagents

  • Gemini 3 Flash Preview3
  • Claude Sonnet 4.62

Scale: 0 to 6 · zero baseline

Portfolios

  • Gemini 3 Flash Preview6
  • Claude Sonnet 4.65
Show the table · 2 rows, 6 columns

Scroll sideways to compare all columns. Select a heading to sort.

Gemini 3 Flash Preview66mixed3645.5m
Claude Sonnet 4.650weak2532m

The published score difference is 16 points (66 versus 50). The leading result's verdict is mixed; the score concerns the whole options-research workflow, rather than account return. The rubric gives 40% weight to fitness, 25% to evidence, 20% to exploration and 15% to risk realism.

The source used the same $25,000 account objective and 25-iteration budget for all eleven runs. Review the actual portfolio and regime evidence before deploying; the leading mixed verdict called for further iteration. This publication has one run per model and no measured billed-cost column.

How the runs worked

Read the test procedure

11 model runs; same task, account context, tools and 25-iteration budget

NexusTrade Aurora autonomous research and options workflow; Agent Run Evaluator V6

  1. Step 1

    Give all 11 models the same watchlist, account context, tools and a 25-iteration budget.

  2. Step 2

    Let each agent inspect market data, explore strategies, backtest and decide whether to deploy.

  3. Step 3

    Grade the complete workflow and the strategy it delivered.

How to read the measurements

  1. Grading and valid output

    The evaluator weights strategy fitness 40%, evidence 25%, exploration 20% and risk realism 15%. No deployment caps fitness at 2/10; single-year evidence caps it at 4/10. The score is a judgment, not a probability of doubling money.

  2. Cost

    This scorecard has no measured billed-dollar cost column.

  3. Time

    Time includes tool use and historical backtesting, not one response.

  4. Subagent

    An agent delegated part of the research to another agent.

  5. Portfolio

    A strategy artifact created during the run. Its existence alone does not establish a successful strategy.

Sample and scope

The publication presents one run per model in this comparison. It does not establish seed-to-seed uncertainty or a general model-quality ranking.

The scorecard and narrative disagree on some subagent/portfolio counts and on how consistently the selected Flash strategy was positive across regimes. The table preserves the published scorecard; conflicting process counts and regime claims are not resolved by inference.

The article describes historical simulations and deployment actions. This study page does not assert independently verified live account returns, achieved doubling, or a deployment recommendation.

Read the original evidence

Aurora product context

Published Aurora workflow recording showing a trading-strategy request and the agent loop
The August article shows this Aurora product workflow. The recording provides product context rather than an individual benchmark response.
See the broader product workflow

This August recording shows the broader Aurora agent workflow. The task explanations above describe each evaluation; its private conversations and complete system prompts remain private.

Continue exploring

Discussion

Sign in or create a free account to join the discussion.

No comments yet.

Aurora · AI research assistant

Try this in Aurora

Edit the request, then open it in Aurora. You choose when to send it.