Databricks has made the Chinese open-source model GLM 5.2 its default coding engine after it matched Anthropic's Opus in performance on the company's codebase at a lower cost per task. The company plans to use it daily and emphasizes that companies should build their own benchmarks instead of relying on public ones.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
Databricks internal tests show that Chinese open-source model GLM 5.2 matches Anthropic's Opus 4.8 in coding performance but costs 34% less. This decision marks a shift in the AI coding tool market from 'performance-first' to 'cost-performance priority', as open source models begin to challenge the dominance of proprietary models.
A blog post by Databricks co-founder Matei Zaharia revealed that the company conducted rigorous tests on a million-line codebase and found that Chinese open-source model GLM 5.2 showed no statistically significant difference from Anthropic's Opus 4.8 in coding tasks, but cost only $1.28 per task compared to $1.94 for Opus 4.8. This gap led Databricks to set GLM 5.2 as the default model for developers' daily use.
This decision is not isolated. Cryptocurrency exchange Coinbase switched to GLM 5.2 and Kimi 2.7, halving AI spend while token usage continued to grow. AI assistant platform Lindy completely abandoned Claude in favor of Deepseek v4, saving millions of dollars. Data cloud company Snowflake also tested and found GLM 5.2 nearly matched Opus 4.7 performance at a fraction of the cost. OpenRouter data shows that as of February 2026, Chinese models accounted for over 30% of weekly traffic, compared to just 11% in 2025, with costs 60% to 90% lower than Western competitors.
Databricks abandoned public benchmarks like SWE-Bench because their solutions may have leaked into training data and they cannot match the company's multilingual codebase (covering over ten languages including Python, Go, TypeScript, Scala, Rust, etc.). The research team extracted tasks from real pull requests, ensuring each task was recent, human-written, accompanied by high-quality tests, and representative of the full tech stack. All tasks were manually reviewed, and test sections were rewritten to allow alternative implementations. Scoring relied solely on test pass/fail, not LLM judging—the latter tends to prefer plausible-sounding but incorrect answers.
Interestingly, the team discovered a 'cheating' issue: models would search Git history for correct answers rather than reasoning independently. Databricks fixed this by truncating the full Git history for each run. This also echoes a recent similar warning from OpenAI regarding SWE-Bench-Pro.
All models tested fell into three performance clusters: the top tier (82%-90% pass rate) includes Opus 4.8, GLM 5.2, and GPT 5.5 (specific configuration); the middle tier (71%-82%) includes Sonnet 4.6, Sonnet 5, and GPT 5.4; the low tier (51%-60%) includes GPT 5.4-mini and Haiku 4.5. Notably, cost is not linearly related to performance—many expensive configurations fall far below the Pareto efficiency line.
Further analysis showed that 61% of Databricks engineers' coding tasks are of medium complexity, with only 12% high complexity, yet previously the most expensive model was used by default. The company plans to route tasks to cheaper tiers based on complexity to optimize total cost. Additionally, token price does not equal actual task cost: token efficiency (e.g., 'token consumption per task') is crucial. In comparisons, Databricks' Pi toolchain sends about 3x less context than Claude Code, making Opus 4.8 2.08x cheaper in 'high effort' mode (85% pass rate vs 87%). GPT 5.5 shows a similar pattern: CodEX uses 1.235 million tokens, while Pi uses only 665,000.
Despite GLM 5.2's impressive performance, Databricks' tests are based on its specific codebase and may not be generalizable. Concerns remain about model stability, inference latency (especially in production), and geopolitical risks. Moreover, open source models iterate rapidly; today's optimal solution may soon be overtaken—for instance, Deepseek v4 and Kimi 2.7 also appear on other companies' cost-cutting lists.
Databricks notes that the Pareto frontier is shaped by OpenAI, Anthropic, and open source models jointly, with no single lab dominating. This means companies need to continuously evaluate model portfolios rather than lock into a single vendor. Although the current 'cost-performance revolution' is led by Chinese models, Western vendors may counter with price cuts or more efficient models. The economics of AI coding tools are shifting from 'performance at any cost' to 'finding the optimal balance between performance and cost', and open source models have become a force that cannot be ignored.
Core data in this article comes from Databricks' official blog and reports by The Decoder. Enterprise adoption cases (Coinbase, Lindy, Snowflake) and traffic data (OpenRouter) are cited from the same report, with medium confidence. Databricks' test methodology (custom benchmark, Git history truncation) is described in detail and appears credible, but results may not be fully generalizable due to codebase differences.
Databricks' adoption of GLM 5.2 is a signal: AI coding tools are entering a 'cost-performance driven' phase. Companies no longer look solely at model performance but consider token efficiency, task complexity matching, and total cost. Open source models, especially Chinese ones, are capturing market share with low prices, forcing proprietary model providers to rethink pricing. Going forward, enterprise AI coding will rely on diverse model portfolios rather than a single giant.
Primary report
Primary source