Teacher forcing
In plain English
Teacher forcing supplies the reference token history so different model variants are scored on the same continuation.
Technical definition
Teacher forcing conditions a model on reference tokens rather than its own sampled outputs, allowing token-level losses to be compared along a fixed sequence.
Engineering details
The article uses a teacher-forced CRUXEval experiment to hold tokens constant while changing Engram and expert routing. This separates changes in reference-answer likelihood from differences caused by models generating different continuations.
Why it matters
Teacher-forced loss and free-generation task accuracy are different measurements. Fixed token histories help isolate an intervention, but they do not measure the entire behavior of a model choosing and extending its own answers.
How to read it in InferenceX
Removing Engram raises answer loss from 0.2848 to 0.3093 bits/token; forcing the ablated model to retain the original expert choices raises it further to 0.3375. These results support partial compensation through rerouting in that experiment, not a universal division between memory and reasoning.