DeepSeek Distillation Doesn't Transfer Censorship: New Evaluation Framework Released

US developers widely use open-source frontier models but worry that foreign actors' (especially the Chinese Communist Party's) values, censorship, or views might transfer along with intelligence gains. CTGT Inc.'s research aims to rigorously test this phenomenon: in a real financial distillation pipeline, using the Chinese censored model DeepSeek V4 Flash as a teacher to train the US model GPT-OSS-120B, observing whether the student learns undesirable behaviors.

It is known that Chinese frontier models noticeably refuse or rewrite answers on sensitive topics, a behavior documented in audits of DeepSeek R1 and V3 series. This study further asks: does the student learn bad behaviors along with skills?

The research team built the LineageEval framework, containing 304 prompts (152 matched pairs). Each pair includes a China-sensitive topic and a structurally identical non-China control topic, e.g., "Great Leap Forward famine" vs. "Ukrainian Holodomor," "Xinjiang labor transfer" vs. "Uzbek cotton," "Tiananmen" vs. "Gwangju." The goal is to distinguish whether a model universally avoids hard questions or specifically avoids China's hard questions.

All 304 responses are scored 0–100 by four independent judges (xAI Grok 4.20, Google Gemini 3.5 Flash, OpenAI GPT-5 Mini, Anthropic Claude Sonnet 4.6), averaged. The matched gap is the sensitive score minus the control score; a positive value indicates selective avoidance of Chinese content. The teacher model is served via self-hosted weights (vLLM) to exclude API-level filtering effects.

On 76 core political prompts, DeepSeek V4 Flash has a matched gap of +45.45 points, with 87% of pairs showing a positive gap. In contrast, the distilled CTGT 120B (Flash-taught) gap is -1.39 points, with only 49% of pairs positive, statistically indistinguishable from the base model GPT-OSS-120B (+0.43 points, 43% positive).

On all 152 pairs (including finance-related), the teacher gap is +32.02, 79% of pairs positive, about 7 standard deviations, p < 0.0001. The three 120B models (base, self-distilled, Flash-taught) have gaps of +3.94, +3.70, and +2.58 respectively, with positive proportions 52–55%, differing by less than 1 point. Distillation does not alter the student model's behavior in unrelated domains.

Under an 8,000 token generation budget, self-distilled CTGT 120B scores 83.61% on FinanceReasoning, higher than Kimi K3 (81.93%) and Inkling (65.13%), with per-query costs 62x lower than Inkling and 160x lower than Kimi K3. Self-distillation (model self-correction) achieves the same score as using the DeepSeek teacher, indicating domain improvement does not depend on the teacher's political bias.

The research team believes many domain-specific applications can gain meaningful improvements without needing a larger teacher model. The self-distilled model matches the Flash-taught model's scores across all random seeds.

The experiment only covers the financial reasoning domain; censorship transfer in other domains (e.g., medical, legal) is unknown. The teacher model is DeepSeek V4 Flash; other Chinese models (e.g., Qwen, GLM) may behave differently.

The base model GPT-OSS-120B itself has a small positive gap of +3.94 on all 152 pairs, source unknown, possibly inherent bias in training data. Distillation does not significantly increase the gap but does not eliminate it entirely.

The study uses four AI judges for scoring, not human experts; the judges themselves may have biases. Human reviewers' scores on individual examples (20 vs 75) align with AI judges (35,30,40,8 vs 72,90,90,74) in trend, but not systematically validated.