How to Fine-Tune an LLM Judge on a Rented GPU. It Has to Beat DeepSeek.

0:00People think training an AI model takes millions of dollars. My first training run is a thirteen-dollar test.

0:06Here's how I'm training an AI that's designed to tackle the world's hardest problem... the stock market.

0:11Recap: I'm building an AI agent that builds profitable trading strategies, and in part two I built the answer key for the critic that checks its work.

0:19Now I'm going to train that critic, using Qwen 3.5 with nine billion parameters.

0:24Here's how I'm doing it, in five steps.

0:26Step one, I turn every example into tokens.

0:30A token is a chunk of a word, the piece a model reads in one bite.

0:34Each example has three parts.

0:36One is the grading rules.

0:38Two is the full agent run, with the agent's whole prompt.

0:42Three is the right answer: the grade, the reason and the evidence.

0:46Glued together, the example on screen is about a hundred thirty-six thousand tokens, and the answer at the end is about two hundred eighty.

0:53Step two, I decide what gets graded.

0:56The model reads the whole run, but only the answer it writes counts toward its score.

1:01Across all the examples, fewer than one token in four hundred gets scored.

1:06Step three, I make the runs fit.

1:08Qwen can read two hundred sixty-two thousand tokens at once.

1:12Eight hundred seventy-nine examples fit, and ten are longer, up to about seven hundred seventy thousand tokens.

1:18I leave those ten out for now.

1:20The same model can stretch to about a million tokens with a technique called YaRN, which stretches how it keeps track of word positions.

1:28But longer reads need a lot more GPU memory, so I'll switch YaRN on only when the finished critic has to grade a really long run.

1:35Step four is training it with a method called LoRA.

1:38Instead of changing all nine billion numbers inside the model, LoRA trains a small set of extra numbers on top.

1:44So why use LoRA? It is far cheaper, and the whole job fits on one GPU.

1:48That's one H200 I rent on RunPod for about five dollars an hour, and the first pilot is capped at about thirteen dollars.

1:55Step five is the race.

1:58My critic will go up against two others: base Qwen with no training, and DeepSeek V4.1 Flash, the best checker from part two.

2:05All three grade the same forty-six hand-checked runs, which the critic never trains on.

2:10Mine wins only if it catches more bad runs than both, without raising more false alarms on good ones.

2:16Bad runs come first, because a bad run that slips through is the one that would let a cheating strategy earn a reward.

2:22I'll save a copy after each round of training and keep whichever one does best on those forty-six runs.

2:28Then comes one final test: a sealed set of runs nobody has touched.

2:33I'll open it once, at the end, so I can't keep tweaking the critic until it passes.

2:38After that, I'll use reinforcement learning to train my own model.

2:42It builds strategies over and over, and each one gets a Sortino score from real live or paper results, or from a walk-forward test when it has none.

2:50And before any score counts, this critic checks how the model got there. If it didn't do what the user asked, or its numbers aren't real, the score is zero.

2:59Today, about one in three strategies my agent builds beats the S&P 500 going forward, and that's the bar.

3:05Like and follow to see if my model clears it.