DeepSeek V4.1 Flash added to AA-AgentPerf benchmark with NVIDIA and AMD results

Artificial Analysis announced that it has added DeepSeek V4.1 Flash to its agentic inference performance benchmark AA-AgentPerf, and simultaneously published results for the model on NVIDIA and AMD hardware. This is the only part of the event that can be treated as established fact: the model was added to the leaderboard, and the results were published.

But all the numbers that follow are different in nature. They come from Artificial Analysis's self-built benchmark and are that organization's self-reported results; no third-party independent verification appears in the source. AA-AgentPerf itself is also described as simulating everyday agentic workloads by replaying real agentic coding sessions (including prompt caching and tool-call latency)—this is a description of the method, not a guarantee of production-environment performance.

So the approach in this article is: treat "who published what" as fact, treat "what the numbers are" as that organization's claims, and treat "what these numbers mean" as inferences that need to be flagged.

Under the condition of 100 tokens/s per user, the throughput ranking given by Artificial Analysis is: GB300 at about 261M tokens/s per megawatt, B300 at about 220M, B200 at about 193M, GB200 at about 178M, and MI355X at about 128M. GB300 takes first place.

This ranking has two preconditions that must be read alongside it. First, the metric is normalized by runtime power per megawatt—that is, throughput divided by power—and the power measurement methodology comes from the organization itself, with no specific measurement method disclosed in the source. Second, the condition is limited to 100 tokens/s per user; with a different concurrency or speed target, the ranking may not hold.

On the AMD side, only one result appears, MI355X; the performance of other AMD hardware is not given in the source, so this leaderboard can only be considered a single point on the AMD dimension, not a line.

At batch size 1—that is, a single agent—all Blackwell systems and MI355X can reach an output speed of about 600 to 670 tokens/s; H100 and H200 reach 372 to 392 tokens/s.

This set of numbers and the throughput ranking in the previous section are talking about two different things. Throughput measures system utilization, while single-agent speed measures the experience of an individual user; the benchmark measures both at once precisely to show "the trade-off between system utilization and individual user experience."

Note that 600 to 670 is a range covering multiple different systems, and the source does not give the specific position of each system within that range, so one cannot conclude from this that there are differences in single-agent speed among the various Blackwell models.

Artificial Analysis says V4.1 Flash can run on as few as 2 GPUs. The inference it draws from this is that the advantage of rack-scale systems over an 8-card single node like B300 becomes smaller.

The specific data supporting this inference is: B300 stays within about 20% of the rack-scale GB300 NVL72 across the entire curve, and leads at both ends. In other words, for this model, the gap bought by paying the rack-scale premium is compressed to within two-tenths—this is the organization's claim, and "about 20%" is an approximation; the precise figure is not given in the source.

There is a causal link here that is easy to skip over: the model being able to run on fewer GPUs is the cause of the change in the hardware comparison landscape. This causal chain comes from the organization's statement, not from an independently verified conclusion.

The background explanation given by the organization states that V4.1 Flash scores 39 on the Intelligence Index, above the earlier V4 Pro 0813's 36, while its parameter scale is about one-third of the latter's: 552B "backbone" parameters and 16B activated parameters, versus the latter's 1.6T and 49B.

Architecturally, it is described as adopting a new causal encoder-decoder design, activating 8B parameters per input token and 16B parameters per output token; it additionally uses a 196B-parameter Engram n-gram memory that can be placed in host memory rather than on accelerators; and its compressed sparse attention requires about one-quarter the per-token KV cache of the earlier V4 Flash model.

These descriptions are Artificial Analysis's background explanation, not official DeepSeek technical documentation. The source gives no first-party provenance for the model weights, architecture details, or official specifications. Therefore numbers such as "parameter scale," "activated parameters," and "about one-quarter KV cache" can only be read as that organization's claims, not as confirmed model specifications.

The organization says these designs make V4.1 Flash easier to serve: weights can be placed on fewer GPUs, and the smaller KV cache leaves room for more concurrent agents per GPU.

Converted into numbers, under the condition of 100 tokens/s per user, V4.1 Flash achieves a throughput-per-megawatt improvement of about 1.3x (GB300) to about 6.2x (B200) compared with its larger sibling model V4 Pro 0813.

This span is worth pausing over: for the same model against the same comparison model, the improvement multiple across different hardware ranges from 1.3x to 6.2x, a difference of nearly fivefold. The source does not explain the reason for this discrepancy. In the absence of an explanation, treating "about 6.2x" as a general efficiency characteristic of the model is a reading that goes beyond the evidence; the more prudent understanding is that the magnitude of improvement is highly dependent on the hardware it is paired with.

Putting the above together, what can be confirmed is: V4.1 Flash was added to AA-AgentPerf, bringing the leaderboard to four featured models in total, the other three being DeepSeek V4 Pro 0813, Kimi K3, and GLM-5.2/5.3; the organization says it will adjust the featured models as new popular open-source models are released, and will provide day-0 results for future major releases.

What cannot be confirmed is more numerous. AA-AgentPerf's specific testing methodology, sample size, and sources of statistical error are not disclosed; the results are based on simulated workloads from replayed sessions and may not be equivalent to production environments; the power normalization relies on the organization's own measurement methodology; and whether the results are reproducible or change over time and with configuration is not stated in the source.

Therefore, the reasonable use of this data is to compare relative positions within the same benchmark, not to treat it as an independent conclusion about V4.1 Flash's serving cost.