TL;DR
Luna's historical next-action billing was $1.67 per 1,000 decisions versus Muse Spark 1.2's $44.23 in the August 8, 2026 replay. Median production latency was 5.6 versus 7.2 seconds. Muse had slightly better valid-output rates, 99.5% versus 99.0%. The judge-dependent means do not establish a reliable quality gap; compare operating savings with the cost of invalid actions in your own task.
Next-action execution model bakeoff: published results
These are exact August 8, 2026 billing cells. Labels are rounded. Muse’s slightly higher output validity and judge-dependent scores remain in the full table and opening summary.
Show the table · 2 rows, 5 columns
Scroll sideways to compare all columns. Select a heading to sort.
| openai/gpt-5.6-luna | 89.2 | 99.0% | $1.67 | 5.6s |
| meta/muse-spark-1.2 | 86.1 | 99.5% | $44.23 | 7.2s |
No matching rows. Clear the filter to see all records.
23-model campaign; 69 frozen decisions × 3 samples; 12 published model rows. Published 2026-08-08.
Which fits the tested job?
Luna had the higher observed next-action score, lower historical bill and lower production p50. Muse Spark 1.2 had slightly better schema validity. That is a real tradeoff if invalid structured actions are costly in your workflow.
Next-action execution model bakeoff
Luna scored 89.2 versus Muse's 86.1 and cost $1.67 versus $44.23 per 1,000 decisions. Its p50 was 5.6s versus 7.2s, while schema validity was 99.0% versus Muse's 99.5%. The judge-dependent means do not establish a reliable quality gap; schema failures and retry costs belong in a task-specific follow-up. These are model results, not a comparison with Meta's personal agent.
Choose the next allowed agent action: follow the published task
A well-formed action can still be the wrong decision. Keep output validity and judged decision quality separate.
Published task summary
The existing plan, conversation and results returned by tools.
Choose one next allowed action, or finish with an answer when the evidence is sufficient.
Read what came back
A structured action. The replay records the proposed action without executing its tools.
Replay 69 saved decisions three times each. Check the required output structure separately from the judge score.
Results from tasks both models attempted
The tested model IDs are openai/gpt-5.6-luna and meta/muse-spark-1.2. Every result table below comes from a study in which both participated.
Compare the measured quality, historical billed cost and elapsed time within the same task and rubric.
Scores from different tasks are not combined into one winner score.
Next-action execution model bakeoff: read these measurements
Read the complete rubric, sample and billing notes
23-model campaign; 69 frozen decisions × 3 samples; 12 published model rows
Choose one next action in an unfinished trading-agent workflow. This replay measures individual decisions, not a fresh complete trading run.
Scoring
Mean action score comes from the judge. Schema validity is the share of actions matching the required output structure; a valid structure can still contain the wrong action. Switching judges reduced the Luna-versus-DeepSeek score gap from 13.9 to 1.5 points.
Cost units
Cost uses historical provider billing per 1,000 decisions. Reused prefixes earned cache discounts; the cached-token split was not retained.
Time units
Prod p50 is median production response latency, separate from replay quality and complete agent wall time.
No comments yet.