Back to feed
News Story
APriority85
机器之心
1 sources

35B Model Outperforms Trillion-Parameter Models! SJTU's AI Starts Creating Its Own Problems and Self-Iterating

A research team from Shanghai Jiao Tong University and other institutions released the scientific research model BigBang-V1, which is post-trained on Qwen3.6-35B-A3B using entirely AI-synthesized data. It achieves the best scores among 35B-scale models on multiple benchmarks, even surpassing some larger models. The key innovation is that AI participates in creating, filtering, and iterating its own training tasks, enabling self-evolution and marking a shift where AI begins to decide what the next generation of models should learn.

SynthePulse Insight · AI deep reading

How a 35B Model Surpasses Trillion-Parameter Giants? SJTU Team Lets AI Create Its Own Problems and Iterate

Version 1 · 1 source

As pretraining data approaches its limits, the next leap in AI may not rely on larger models but on letting AI participate in creating its own training tasks. BigBang-V1, released by Shanghai Jiao Tong University and others, uses about 10,000 AI-synthesized data points to surpass the 1.6T-parameter DeepSeek V4 Pro Preview on multiple benchmarks. The underlying data self-evolution mechanism may herald a new direction in AI research.

  • BigBang-V1 is post-trained on Qwen3.6-35B-A3B, with training data entirely AI-synthesized, totaling only about 10,000 entries.
  • Among 35B-scale models, BigBang-V1 achieves the highest score on 10 evaluations, and surpasses the 1.6T-parameter DeepSeek V4 Pro Preview on tasks like FrontierScience Research.
  • The BigBang framework includes inner and outer loops: the inner loop has Generator and Critic Agents continuously generating and filtering tasks, while the outer loop calibrates data value using real-task evaluations.
  • The team calls this 'early recursive self-improvement at the data level,' but full RSI still requires AI to autonomously improve its R&D capabilities.
Open section navigationData Bottleneck: Human Expert Problem Creation Can't Keep Up with Model Consumption

Data Bottleneck: Human Expert Problem Creation Can't Keep Up with Model Consumption

Ilya Sutskever once pointed out that pretraining data will eventually be exhausted, and simply scaling up training by 100 times won't lead to fundamental changes. As model capabilities approach human expert levels, the question of where the next batch of truly valuable training tasks will come from becomes critical.

What models lack isn't just problems, but 'problems plus feedback': tasks that are sufficiently frontier and verifiable to form stable training signals. Science, offering multiple verification methods like formal proofs and program execution, becomes the intersection BigBang chose.

The BigBang Framework: Letting AI Participate in Problem Creation, Review, and Iteration

BigBang's technical framework includes inner and outer loops. The inner loop consists of a Generator Agent and a Critic Agent: the Generator works as a code agent, able to modify, execute, and debug data synthesis programs; the Critic handles two-level review, first checking basic constraints, then evaluating higher-order quality.

The outer loop calibrates the Critic's bias: the system trains models with different versions of the data pipeline, evaluates them on real research tasks, and a Meta-Critic compares the Critic's predictions with actual capability changes, adjusting the next round of synthesis strategies.

This flywheel mechanism allows the data production system to co-evolve with model capabilities, optimizing not just individual samples but the entire data production program.

Experimental Results: Small Model Crosses Scale, Capabilities Transfer to Search and Code

BigBang-V1's post-training uses about 10,000 AI-synthesized data entries, surpassing the base model on 10 out of 11 evaluations. It scores 46.2 on FrontierScience Research, exceeding DeepSeek V4 Pro Preview's 40.7; on Humanity's Last Exam, it scores 50.3 versus 48.2.

On BrowseComp, it achieves 76.5, and on SWE-Bench Pro, 54.2, both significantly higher than the base model. The team believes the simultaneous improvement in search and code capabilities indicates the model learned a cross-domain problem-solving process, not just scientific knowledge.

Case studies show that in a virus identification task, BigBang-V1 consistently converged to Norovirus across three runs, while DeepSeek V4 Flash Preview gave three different answers; in an elliptic integral task, BigBang-V1 completed a full analytical derivation, whereas DeepSeek V4 Pro Preview, though correct, relied on existing functions.

Positioning and Limitations: Early Recursive Self-Improvement at the Data Level

BigBang is not a precedent; Absolute Zero and AlphaEvolve have already let AI autonomously propose verifiable tasks. BigBang's difference lies in jointly optimizing the task generation program, distribution, and evaluation criteria, and calibrating the synthesis pipeline with real-task gains.

The paper calls it 'early recursive self-improvement at the data level.' Full RSI typically refers to AI autonomously improving its R&D capabilities and training stronger successor models; BigBang currently demonstrates co-evolution of the data pipeline and model capabilities, a phased approach.

Credibility boundary

This article's information primarily comes from a report by Machine Intelligence, a secondary source. All performance data and model details are from that report and have not been independently verified. Some conclusions (such as capability transfer) are the team's inferences and should be treated with caution.

Insight takeaway

BigBang-V1 demonstrates the possibility of significantly improving capabilities without scaling model size, solely by changing how training data is produced. Its data self-evolution mechanism offers a new path for the post-Scaling Law era, but it remains distant from full RSI.

Sources for this version

  1. 35B干赢万亿参数大模型！上交大AI开始给自己造题还能自我迭代了机器之心

Primary report

机器之心

Primary source