TL;DR
Gemini 3.5 Flash answered 72.7% of 22 stock questions successfully in the August 8, 2026 SQL study, with a blended score of 0.727 and average workflow time of 33.3 seconds. Gemini 3.6 Flash scored higher in the same comparison. The blended score includes non-answer zeros; it measures this data-query workload, not trading returns or correctness only among graded answers.
- Exact tested ID
- google/gemini-3.5-flash
- Published studies
- 1
- Provider
Models answering 22 stock questions with SQL: compare the tested outputs
These are the exact August 8, 2026 success-rate cells for 22 questions. Labels are rounded; the full table keeps blended scores and workflow times. Success rate and correctness among graded answers are different measures.
Show the table · 2 rows, 5 columns
Scroll sideways to compare all columns. Select a heading to sort.
| google/gemini-3.5-flash | 0.727 | 1.00 | 72.7% | 33.3s |
| google/gemini-3.6-flash | 0.841 | 1.00 | 86.4% | 26.6s |
No matching rows. Clear the filter to see all records.
Original units preserved. Average score is the historical blended measure; this table is not a new run.
google/gemini-3.5-flash was tested on 2026-08-08. 22 natural-language questions; 6 published model rows. Its row and the other published arm use the same task and measurement units.
What this version helps you choose
Its stock-query blended score and success rate were below Gemini 3.6 Flash’s in the same 22-question comparison.
Answer a stock question with a data query: follow the published task
This published example explains the task. It is not an identified scored fixture or an answer attributed to any one model. Stored P/E and price divided by the displayed EPS are different fields in its query.
Published worked example, without model attribution
What were the trailing P/E ratios for TSLA and GOOGL on 2025-05-21?
Translate the question into DuckDB SQL, a query over the financial data, then inspect the returned rows and explain the answer.
Read what came back
The original example reported GOOGL at $168.56 with stored P/E 18.46, and TSLA at $334.62 with stored P/E 169.17. Both rows show 2025-05-21.
Inspect both tickers, the requested date and four distinct fiscal quarters of earnings. The original EPS query did not deduplicate fiscal periods, and the correction has not been rerun.
Models answering 22 stock questions with SQL
Inspect the original Gemini 3.5 metrics
On 22 stock questions, Gemini 3.5 Flash recorded a blended score of 0.727, success rate 72.7% and average workflow time 33.3s. The blended score includes zeros for non-answers; it is different from correctness among answers that reached grading. The example below explains the data task without attributing its answer to this model.
| Published metric | Value |
|---|---|
| Average score | 0.727 |
| Median | 1.00 |
| Success rate | 72.7% |
| Avg execution | 33.3s |
No matching rows. Clear the filter to see all records.
Models answering 22 stock questions with SQL: read these measurements
Read the complete rubric, sample and billing notes
22 natural-language questions; 6 published model rows
Answer financial questions by writing and executing DuckDB SQL against the stock data lake.
Scoring
The average is a blended score across all question cells, including zeros for clarification or execution failure. Success rate and graded-only correctness answer different questions; none measures investment returns.
Cost units
The published screening table has no per-model cost column.
Time units
Average execution time covers the SQL-answer workflow, not the executor study's production p50.
Graded-only mean
The mean among answers that reached grading. Luna recorded 0.931 across 13 graded cells; nine ungraded cells lowered its blended score to 0.550.
TTM earnings / P/E
Trailing twelve-month earnings cover four fiscal quarters. Price-to-earnings relates price to earnings; the published query reads a stored P/E field rather than recomputing it from the displayed EPS.
No comments yet.