Research library

Model evaluations

Compare published AI model results by task, exact version, grading method, cost and latency. Start with the benchmark that matches your workload.

Model evaluations

Recommended starting point

Models answering 22 stock questions with SQL

Published 2026-08-08: 22 natural-language questions; 6 published model rows. Average answer score, success rate and average execution time. See the results, how they were measured and where the findings stop.

Open page
Recommended starting point

Best AI models for Stock Screening: published evaluations

Which models performed well on stock screening in NexusTrade's published evaluations? Compare the task, exact versions, costs and evidence limitations.

Open page

AI model benchmarks: every published evaluation by date

Browse 24 dated AI model evaluations, covering 85 exact model versions. Each study lists its task, results and method.

Open page

Artifact grading and repeatability study

See ten filing transactions become nineteen extracted rows, then follow how independent reviewers reduce their findings. Compare the actual published recall and consistency results.

Open page

Best AI models for Agent Execution: published evaluations

Which models performed well on agent execution in NexusTrade's published evaluations? Compare the task, exact versions, costs and evidence limitations.

Open page

Best AI models for Agent Planning: published evaluations

Which models performed well on agent planning in NexusTrade's published evaluations? Compare the task, exact versions, costs and evidence limitations.

Open page

Best AI models for algorithmic trading: evidence by task

Compare published model evaluations for options strategies, planning, agent execution, artifact review, sandbox code and stock SQL screening. Each task keeps its own methodology.

Open page

Best AI models for Artifact Review: published evaluations

Which models performed well on artifact review in NexusTrade's published evaluations? Compare the task, exact versions, costs and evidence limitations.

Open page

Best AI models for Options Strategy Development: published evaluations

Which models performed well on options strategy development in NexusTrade's published evaluations? Compare the task, exact versions, costs and evidence limitations.

Open page

Best AI models for Sandbox Code Generation: published evaluations

Which models performed well on sandbox code generation in NexusTrade's published evaluations? Compare the task, exact versions, costs and evidence limitations.

Open page

Best AI voice model in 2026: HeyGen Voice 1 Professional

Published 2026-10-16: Two HeyGen Voice 1 voice clones created on 2026-10-14: a Professional Clone trained on 72m32s of speech and an Instant Clone from one 2m50s recording, each with one checked demo. Which clone the author approved and now uses by default, with each demo's duration and format. See the results, how they were measured and where the findings stop.

Open page

Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs

Published 2024-06-25: 2 portfolio-creation prompts, 2 models, single run each. Author-graded correctness of the generated portfolio configuration, approximate time to the final response, and list price per million tokens. See the results, how they were measured and where the findings stop.

Open page

Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query

Published 2024-06-25: 1 screening request, 2 models, single run each. Whether the model returned an answer, and the time to that answer. See the results, how they were measured and where the findings stop.

Open page

Claude 3.5 Sonnet and o3-mini building a Bitcoin trading strategy

Published 2025-02-07: 1 strategy request, 2 models, one attempt each. Whether the generated buy rule matched the requested condition, judged by the author. See the results, how they were measured and where the findings stop.

Open page

Claude 3.5 Sonnet: trading task evaluations

See how anthropic/claude-3.5-sonnet performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Claude 3.7 Sonnet and o3-mini building a vague leveraged-ETF strategy

Published 2025-02-24: 1 ambiguous strategy request, 2 models, one generated strategy each, backtested once. Backtest statistics of each generated strategy: percent change, Sharpe ratio, Sortino ratio, max and average drawdown, number of trades. See the results, how they were measured and where the findings stop.

Open page

Claude 3.7 Sonnet: trading task evaluations

See how anthropic/claude-3.7-sonnet performed in 4 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Claude Opus 4: trading task evaluations

See how anthropic/claude-opus-4 performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Claude Opus 4.1: trading task evaluations

See how anthropic/claude-opus-4.1 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Claude Opus 5: trading task evaluations

See how anthropic/claude-opus-5 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Claude Sonnet 4: trading task evaluations

See how anthropic/claude-sonnet-4 performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Claude Sonnet 4.6: trading task evaluations

See how anthropic/claude-sonnet-4.6 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Claude Sonnet 5: trading task evaluations

See how anthropic/claude-sonnet-5 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

DeepSeek R1 and o3-mini writing a dividend stock screen in SQL

Published 2025-02-01: 1 natural-language screening request, 2 models, R1 asked twice, o3-mini once. Author-graded accuracy (0 or 1), estimated cost in units of one R1 request (c), and wall-clock latency. See the results, how they were measured and where the findings stop.

Open page

DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL

Published 2025-01-21: 3 chained SPY research questions in one conversation per model, 2 models, single run each. Answer values each model returned per question, the article's correctness verdict on question 1, and list price per million tokens. See the results, how they were measured and where the findings stop.

Open page

DeepSeek R1: trading task evaluations

See how deepseek/deepseek-r1 performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

DeepSeek V3: trading task evaluations

See how deepseek/deepseek-chat performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Deepseek V4 Flash: trading task evaluations

See how deepseek/deepseek-v4-flash performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Deepseek V4 Pro: trading task evaluations

See how deepseek/deepseek-v4-pro performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Eleven models building an options strategy

Published 2026-04-06: 11 model runs; same task, account context, tools and 25-iteration budget. Goal-oriented evaluator score / 100. See the results, how they were measured and where the findings stop.

Open page

Five models completing one sandbox research task

Compare every reported grade for five models working on one real sandbox research task. See the regressions, recorded costs and the self-check mistake that a successful run can hide.

Open page

Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL

Published 2025-02-05: 2 natural-language financial questions, 3 models, one attempt each. Author-assigned score per question (0 to 1) and published OpenRouter list price per million tokens. See the results, how they were measured and where the findings stop.

Open page

Gemini 2.0 Flash: trading task evaluations

See how google/gemini-2.0-flash-001 performed in 4 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemini 2.5 Flash Preview: trading task evaluations

See how google/gemini-2.5-flash-preview performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Gemini 2.5 Flash: trading task evaluations

See how google/gemini-2.5-flash performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemini 2.5 Pro Preview: trading task evaluations

See how google/gemini-2.5-pro-preview performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Gemini 2.5 Pro: trading task evaluations

See how google/gemini-2.5-pro performed in 4 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemini 3 Flash Preview vs Claude Sonnet 4.6: options-strategy development

Compare exact model versions on 1 shared published evaluation of options-strategy development. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

Gemini 3 Flash Preview vs GPT-5.4: options-strategy development

Compare exact model versions on 1 shared published evaluation of options-strategy development. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

Gemini 3 Flash Preview vs Grok 4.20: options-strategy development

Compare exact model versions on 1 shared published evaluation of options-strategy development. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

Gemini 3 Flash Preview: trading task evaluations

See how google/gemini-3-flash-preview performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemini 3 Pro Preview: trading task evaluations

See how google/gemini-3-pro-preview performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Gemini 3.1 Flash Lite: trading task evaluations

See how google/gemini-3.1-flash-lite performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemini 3.5 Flash Lite: trading task evaluations

See how google/gemini-3.5-flash-lite performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemini 3.5 Flash: trading task evaluations

See how google/gemini-3.5-flash performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Gemini 3.6 Flash: trading task evaluations

See how google/gemini-3.6-flash performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Gemma 4 31B: trading task evaluations

See how google/gemma-4-31b performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GLM 5.2: trading task evaluations

See how z-ai/glm-5.2 performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT 5 Mini: trading task evaluations

See how openai/gpt-5-mini performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT 5.6 Luna Pro: trading task evaluations

See how openai/gpt-5.6-luna-pro performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT 5.6 Luna vs Claude Opus 5: agent planning

Compare exact model versions on 1 shared published evaluation of agent planning. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs Claude Sonnet 5: agent planning

Compare exact model versions on 1 shared published evaluation of agent planning. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs Deepseek V4 Flash: planning, execution and sandbox research

Compare exact model versions on 3 shared published evaluations of planning, execution and sandbox research. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs Gemini 3 Flash Preview: agent execution and stock-screening SQL

Compare exact model versions on 2 shared published evaluations of agent execution and stock-screening SQL. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs Gemini 3.6 Flash: planning, execution and stock-screening SQL

Compare exact model versions on 3 shared published evaluations of planning, execution and stock-screening SQL. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs GLM 5.2: planning, execution and sandbox research

Compare exact model versions on 3 shared published evaluations of planning, execution and sandbox research. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs MiniMax M3: artifact review

Compare exact model versions on 1 shared published evaluation of artifact review. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs Muse Spark 1.1: agent planning

Compare exact model versions on 1 shared published evaluation of agent planning. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna vs Muse Spark 1.2: agent execution

Compare exact model versions on 1 shared published evaluation of agent execution. See the observed tradeoffs, rubric, sample, date and cost units.

Open page

GPT 5.6 Luna: trading task evaluations

See how openai/gpt-5.6-luna performed in 5 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT 5.6 Terra Pro: trading task evaluations

See how openai/gpt-5.6-terra-pro performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-3.5 Turbo: trading task evaluations

See how openai/gpt-3.5-turbo performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT-4.1 Mini: trading task evaluations

See how openai/gpt-4.1-mini performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-4.1 Nano: trading task evaluations

See how openai/gpt-4.1-nano performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-4.1: trading task evaluations

See how openai/gpt-4.1 performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT-4o mini and GPT-3.5 Turbo generating a portfolio JSON object

Published 2024-07-19: 1 portfolio JSON task, 2 models, prompt count and run count not published in this excerpt. Pass or fail: whether the model produced a valid portfolio JSON object that worked downstream. See the results, how they were measured and where the findings stop.

Open page

GPT-4o mini and GPT-3.5 Turbo generating a valid SQL query

Published 2024-07-19: 1 SQL generation task, 2 models, prompt count and run count not published in this excerpt. Pass or fail: whether the model produced a syntactically valid query that executed and returned results the author judged correct. See the results, how they were measured and where the findings stop.

Open page

GPT-4o mini: trading task evaluations

See how openai/gpt-4o-mini performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT-4o: trading task evaluations

See how openai/gpt-4o performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT-5 Chat: trading task evaluations

See how openai/gpt-5-chat performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-5 family and peers generating financial SQL queries

Published 2025-08-10: EvaluateGPT financial question set (count not restated in this article), 8 models. Median and average judge score (0 to 1), success rate, average response time and list token prices. See the results, how they were measured and where the findings stop.

Open page

GPT-5 Nano: trading task evaluations

See how openai/gpt-5-nano performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-5: trading task evaluations

See how openai/gpt-5 performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

GPT-5.4-mini: trading task evaluations

See how openai/gpt-5.4-mini performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-5.4: trading task evaluations

See how openai/gpt-5.4 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

GPT-oss and Gemini generating financial SQL queries

Published 2025-08-06: 90 financial questions per the article text, 4 models. Median and average judge score (0 to 1), success rate, average response time, list token prices and context window. See the results, how they were measured and where the findings stop.

Open page

gpt-oss-120b: trading task evaluations

See how openai/gpt-oss-120b performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

gpt-oss-20b: trading task evaluations

See how openai/gpt-oss-20b performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Grok 3 Beta: trading task evaluations

See how x-ai/grok-3-beta performed in 4 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Grok 3 Mini Beta: trading task evaluations

See how x-ai/grok-3-mini-beta performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Grok 4: trading task evaluations

See how x-ai/grok-4 performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Grok 4.20: trading task evaluations

See how x-ai/grok-4.20 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Grok 4.5: trading task evaluations

See how x-ai/grok-4.5 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Grok Build 0.1: trading task evaluations

See how x-ai/grok-build-0.1 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Hy3: trading task evaluations

See how tencent/hy3 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Jev 1.14 vs GPT-5.6 Luna: mapper and reducer benchmark

We reviewed saved Jev 1.14 and GPT-5.6 Luna responses separately as mappers and reducers. See defect coverage, final verdicts and a replay of false flags.

Open page

Kimi K2.5: trading task evaluations

See how moonshotai/kimi-k2.5 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Kimi K3: trading task evaluations

See how moonshotai/kimi-k3 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Laguna Xs 2.1: trading task evaluations

See how poolside/laguna-xs-2.1 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Llama 4 Maverick: trading task evaluations

See how meta-llama/llama-4-maverick performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

MiMo v2 Pro: trading task evaluations

See how xiaomi/mimo-v2-pro performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

MiniMax M3: trading task evaluations

See how minimax/minimax-m3 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Mistral Small 2603: trading task evaluations

See how mistralai/mistral-small-2603 performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Models generating financial SQL queries, July 2025

Published 2025-07-11: About 88 financial questions per the article text, 11 models. Median and average judge score (0 to 1), success rate, average response time and list token prices. See the results, how they were measured and where the findings stop.

Open page

Models generating SQL for financial questions, April 2025

Published 2025-04-15: 60 financial questions per the article text, 11 models. Average, median and standard deviation of query score. Success rate. Average execution time. List input and output price. See the results, how they were measured and where the findings stop.

Open page

Models turning plain-English trading rules into strategy configs

Published 2025-08-11: 14 models on one sample of strategy requests, this article does not state the count (the companion article of 2025-08-12 describes the same table as 12 financial questions). Median and average semantic judge score (0 to 1), success rate, average generation time in milliseconds, and list input/output price per million tokens. See the results, how they were measured and where the findings stop.

Open page

MongoDB time-series collections vs TimescaleDB for backtest data

Published 2025-10-03: 2 databases, backtest and portfolio-history queries, 20 requests in batches of four per the article. Total storage, average backtest and portfolio query latency, combined latency, and monthly cost. See the results, how they were measured and where the findings stop.

Open page

Muse Spark 1.1: trading task evaluations

See how meta/muse-spark-1.1 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Muse Spark 1.2: trading task evaluations

See how meta/muse-spark-1.2 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Nemotron 3 Ultra 550B: trading task evaluations

See how nvidia/nemotron-3-ultra-550b performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Nemotron 3.5 Super VL benchmark: 298 trading-assistant attempts

298 attempts, four full suites, and saved incumbents for comparison. See the grades, recorded costs and response times.

Open page

Next-action execution model bakeoff

Published 2026-08-08: 23-model campaign; 69 frozen decisions × 3 samples; 12 published model rows. Mean next-action score; schema validity, billed cost and production p50 are separate. See the results, how they were measured and where the findings stop.

Open page

o1: trading task evaluations

See how openai/o1 performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

o3-mini: trading task evaluations

See how openai/o3-mini performed in 6 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

o3: trading task evaluations

See how openai/o3 performed in 2 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

o4-mini: trading task evaluations

See how openai/o4-mini performed in 3 trading task evaluations, with exact version, measured results, dates and test inputs.

Open page

Planning models on 29 frozen conversations

Published 2026-08-08: 16-model campaign, 29 frozen conversations; 14 model rows shown in the public table. Mean graded plan score; standard error reported separately. See the results, how they were measured and where the findings stop.

Open page

Qwen3.7 Flash: trading task evaluations

See how qwen/qwen3.7-flash performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

Qwen3.8 Max: trading task evaluations

See how qwen/qwen3.8-max performed in 1 trading task evaluation, with exact version, measured results, dates and test inputs.

Open page

ScaleDown Extract Review: Accuracy, Speed and Cost Tested

ScaleDown Extract got GAAP EPS right on 86 of 138 real 8-K earnings releases, against 121 for DeepSeek V4 Flash. It was 12x faster and about 10x cheaper.

Open page

ScaleDown Summarize Review: Accuracy, Speed and Cost Tested

ScaleDown Summarize was accurate on 11 of 12 real documents and 15x faster than Muse Spark 1.3, but misread 6 of 13 stock screener tables.

Open page

Seven models generating financial SQL queries

Published 2025-04-10: 7 models on a shared SQL question set, the article does not state the number of questions. Per-query score from 0 to 1 summarized as average, median and standard deviation, plus success rate, average execution time in milliseconds and share of perfect scores. See the results, how they were measured and where the findings stop.

Open page

Voice-cloning systems ranked blind on one speaker's voice

Published 2026-08-24: 16 ranked arms (fine-tuned and zero-shot variants of what the article calls eleven systems), each reading the same 67-word script, plus a recorded human control. Blind rank by perceived similarity to the author's voice, with the author's verbatim verdict and any reported clip duration or word coverage. See the results, how they were measured and where the findings stop.

Open page

Continue exploring

Browse the complete library ↗