Phenx
← All research
Learning & evaluation·September 13, 2026·7 min read

I Sent a Small AI Model to School. Here’s Its Report Card.

A bigger AI taught it through 26 puzzles. Then it had to sit the exam alone.

School-year experiment: unseen correct attempts were 8 of 78 for the frozen model and 25 of 78 for the trained student; one 4B student, one 14B teacher, and 26 puzzles.

Saurabh Sarkar · Phenx Machine Learning Technologies · September 2026

My student finished its school year with a 32% score on questions it hadn’t practiced. An unchanged copy of the same model took the same exam and scored 10%.

I wouldn’t call either score good. But the gap suggested that some of the lessons had stuck, even when the teacher was no longer there to help.

The student was a small language model. I gave it a larger model as a teacher and a curriculum of puzzles, with practice between subjects and reviews along the way. The whole school year took about seven hours.

I wanted a correction to be useful beyond the conversation in which it happened. If the student made a mistake and received a lesson, could it handle a different problem later without the teacher or the old explanation in its prompt?

The student, Qwen3-4B, had four billion parameters. Its teacher, Qwen3-14B, had fourteen billion. The timetable covered Sudoku, shortest paths, dice probability, binary grids, course prerequisites, and cube rotation.

I started with a fresh student and let it keep what it learned as it moved between subjects. A probability lesson could build on the same trained model that had just worked through a grid puzzle.

Those changes went into a rank-16 LoRA adapter, a set of extra trainable parameters attached to the model. The original weights stayed fixed. Switching the adapter off gave me the unchanged student for comparison; switching it back on restored the learning.

The teacher read the student’s written work, looked for mistakes, and chose exercises. Computer solvers supplied the worked answers. Dice probabilities were calculated exactly, routes were checked for the shortest path, and grids had to satisfy their rules. If a puzzle had more than one valid solution, a different answer could still pass.

Each lesson gave the student four worked examples, followed by two fresh practice questions. Two successful practice rounds in a row let it move on early. Otherwise, teaching stopped after ten rounds.

I kept the original answer out of training. The lessons used nearby problems with changed numbers or constraints, and the practice and final-test answers stayed out too. That reduced the opportunity to repeat a rehearsed answer, though the student could still learn a narrow pattern.

New material arrived with reminders of earlier lessons. Three quarters of each training update’s loss came from the current example and one quarter from previously taught examples.

After ten puzzles, the student sat a review covering all ten. I repeated that after twenty and again after all twenty-six. Anything it couldn’t solve on all three attempts earned it extra lessons, capped at three more teaching rounds.

Then it took the whole review again, including the questions it had already passed. I wanted to know whether fixing one weakness disturbed something it could previously do.

By the end, the student had received 40 extra teaching sessions on top of the original 26. All reviews and final tests were unaided, and no training happened during a test.

The final exam used questions held back from the lessons. Once it began, teaching was over. I didn’t use the results to give another lesson or choose a better-performing version of the student.

There were 26 unseen benchmark questions, each attempted three times. The unchanged model faced the same questions with the same answer limits and matched random seeds.

Correct attempts on 26 questions, with three attempts per model.
EvaluationUnchanged modelTrained student
Unseen benchmark questions8/78 · 10.3%25/78 · 32.1%
Original development problems6/78 · 7.7%41/78 · 52.6%
Separate generated reference variants18/78 · 23.1%40/78 · 51.3%

Each row contains 26 questions, with three attempts per model. The scores count correct attempts, not different questions solved.

The originals had already helped the teacher decide where the student needed work. Their answers weren’t used for training, but they were familiar to the teaching process. The unseen benchmark questions were the stronger test of whether a lesson could carry over.

The generated reference variants formed a separate test set. The experiment called this “retention,” although all testing happened at the end of the run. It didn’t measure what the student would remember weeks later.

Final report card: trained versus frozen scores were 25 versus 8 of 78 on unseen questions, 41 versus 6 on originals, and 40 versus 18 on generated reference variants.

Seven unseen questions came back correct on all three attempts. The unchanged model managed that on none. At least one attempt succeeded on ten questions for the trained student, compared with six for the unchanged model.

That left sixteen unseen questions where the student failed every attempt. Much of the exam remained unresolved.

Sudoku looked like one of the student’s stronger subjects until I compared the different papers.

It got 22 of 30 attempts right on the original Sudoku problems and 21 of 30 on generated reference variants. On unseen benchmark Sudoku questions, it got four of thirty.

The new questions may have been harder, or may have required steps the generated lessons didn’t cover. Similar numbers of filled-in cells don’t make two Sudoku puzzles equally difficult. The student may also have specialized in a narrow kind of puzzle. I can’t distinguish those explanations from this run.

That makes me reluctant to prescribe more of the same homework. I’d first look at what separates those four successes from the twenty-six failures.

Other subjects gave a mixed report card. Dice follow-ups scored five out of nine attempts against zero for the unchanged model. Binary grids tied at three out of nine. Cube rotation scored three out of three, but that was one question answered three times. It’s too small a sample to declare a favorite subject.

Sudoku results for the trained student: 22 of 30 attempts correct on originals, 21 of 30 on generated reference variants, and 4 of 30 on unseen benchmark follow-ups.

The final round of extra lessons was followed by a worse score on the original problems: 41 correct attempts out of 78, down from 46.

Two previously passed questions, one binary grid and one shortest path, fell from three correct attempts to zero. Neither had received extra lessons in that review because both had passed it.

Later training may have interfered with earlier learning. But the tests also used different random seeds, and the unchanged model’s score moved from three to six without any training at all. To separate those effects, I’d need to give the pre-review student another test using the after-review seeds.

Retesting the passed questions exposed those two drops. A review limited to the student’s known weaknesses would have missed them.

Final review: trained original scores changed from 46 to 41 of 78; frozen scores changed from 3 to 6. Different sampling seeds prevent attributing the decline solely to teaching.

The school year combined continued training, reminders of old lessons, and extra help after mistakes. Together, they produced a better score on unseen questions in this run. I can’t yet say how much each part contributed, or whether a fixed set of lessons would have worked as well without the teacher choosing them.

This was one student training run, in one curriculum order, on a small set of puzzles that had already helped shape the experiment. Three attempts per question tell me about answer consistency; they don’t replace training another fresh student and seeing whether the result holds.

The grading also stops short of proving that every explanation was sound. For course-prerequisite puzzles, for example, a correct yes-or-no answer could pass without a valid explanation. And “unseen” means held out of our lessons, not guaranteed absent from the model’s original pretraining.

I’d compare teacher-selected lessons with a fixed curriculum on the same new questions and comparable training budgets, then repeat with fresh students. For Sudoku, I’d examine the lesson set before adding to it. It could handle most of the originals and generated variants while missing nearly all the benchmark follow-ups. I’d want to understand that mismatch before giving it another stack of homework.

The experiment record contains the full protocol, per-question scores, and September 13 audit. The run used 2,848 optimizer updates and produced 984 review and final-test answers. The research history covers earlier versions, whose different final tests prevent a direct comparison with this one.