Rewarding the Answer Doesn’t Specify the Reasoning Language
A correct solution and a readable explanation are different training goals.
If you reward a model for getting the answer right, you haven’t necessarily rewarded it for explaining every step in English.
That helps explain something interesting about bilingual reasoning models: they can switch between English and Chinese while working through a problem. A reward based on the final answer can favor a successful solution even when the intermediate steps mix languages. Whether the explanation reads naturally to an English speaker is a separate condition.
DeepSeek-R1-Zero developed language mixing during reinforcement learning. When the researchers developed R1, they added a reward for language consistency, measured by the proportion of words in the reasoning that used the target language. They reported a slight performance reduction from that constraint, but chose it to make the reasoning more readable.
I think “path of least resistance” is a useful way of thinking about this, provided we’re clear about what the resistance is. Training makes successful patterns more likely. If a bilingual model has learned useful reasoning patterns in both languages, an outcome reward doesn’t automatically give it a reason to keep the whole explanation in just one.
That doesn’t require Chinese to be inherently cheaper in tokens. Getting the answer right and keeping the reasoning in a particular language are two things you can ask the training process to care about separately.