OpenDesign DeepSeek Flash Eco: Where It Differs

DeepSeek Flash Eco vs DeepSeek Flash benchmark on Open Design
この記事は約11分で読めます。

OpenDesign Go added deepseek-v4.1-flash-eco. The official pitch is "for non-urgent work, no quality trade-off." I sent 960 requests to see how far that claim holds.

Key Takeaways

  • The no-trade-off claim holds: code execution tests passed 18/18 for both
  • Multimodal holds too: images worked despite the catalog listing text only
  • The delay is uneven: no gap on short turns, 2x on generation, 6x on long inputs
  • Different upstreams: Volcengine for Flash, BytePlus for Eco

The quality claim holds

Eco is positioned for non-urgent work. That leaves one practical question: how long is the wait, and on which tasks?

In my measurements, the answer depended almost entirely on input length.

  • Quality: identical on every checkable task I could build
  • Short single turn: no measurable difference in time to first token
  • Streaming output: about half the speed
  • Long documents: up to six times slower

Checking the marketing claims against measurements

I lined up each claim with what I measured. Every quality claim held up.

ClaimMeasuredVerdict
Same weights as V4.1 FlashDifferent upstream, near-identical qualityBroadly holds
No quality trade-offCode 18/18, reasoning 9/10, images 7/8Holds
Multimodal support7 of 8 image tasks correctHolds
Full 1M contextAccepted 920k tokensHolds
For non-urgent workSame speed on short turns, 6x slower on long inputsShape holds

The interesting part is that "non-urgent" is doing a lot of work in that sentence. It covers a range from no difference at all to a one-minute wait per request.

They are not even running on the same upstream

The first thing I checked was whether both models hit the same backend.

Model IDs confirmed:

id: deepseek-v4.1-flash       input_modalities: ["text","image"]
id: deepseek-v4.1-flash-eco   input_modalities: ["text"]

The stream also reports routing_info. Across ten attempts each, the routing never changed.

ModelUpstreamActual model ID
deepseek-v4.1-flashcustom_provider_volcengine_speeddeepseek-v4-1-flash-261017
deepseek-v4.1-flash-ecocustom_provider_byteplusdeepseek-v4-1-flash-economy-261017

Different providers entirely, not a quantization variant of the same model. That explains the speed gap.

Decoding speed is where the 2x lives

I disabled thinking with reasoning_effort=none to isolate raw generation speed.

{"model": "deepseek-v4.1-flash-eco", "max_tokens": 4000,
 "reasoning_effort": "none",
 "messages": [{"role": "user", "content": "Count from 1 to 300"}]}

Counting to 300, five runs each, interleaved between models. Medians:

MetricflashecoRatio
Time to first token1.43 s1.38 s0.96x
Decode speed288.7 tok/s139.5 tok/s0.48x
Total for 898 tokens4.85 s7.82 s1.61x

The first token arrives at the same time. Everything after that is half speed.

Long inputs make the gap much worse

Input prefill is the real bottleneck. Each run used a unique prefix so caching could not help.

Input tokensflashecoRatio
8,0002.32 s3.23 s1.39x
32,0003.03 s8.61 s2.84x
100,0005.37 s24.33 s4.53x
200,0009.94 s52.93 s5.33x
400,00016.98 s105.05 s6.19x

The ratio grows with input length. For reading long documents there is no reason to pick eco.

Both models accepted 920k tokens, close to the advertised 1,048,000 limit.

Quality was essentially a tie

I used only tasks with checkable answers. Two entries my first scoring script marked wrong were corrected after reading the raw output.

TestDetailflasheco
Code generationLRU, parser, interval merge, 5 repeats each18/1818/18
Reasoning and math10 single-answer problems9/109/10
Image understandinginvoice, bar chart, code screenshot, typo spotting7/87/8
Instruction followinglength, JSON, bullets, banned words, table4/55/5
Hallucination resistancefake API, fake paper, uncertainty2/32/3

Code output was actually executed against 15 test suites. Both passed everything.

Only one instruction-following case differed. Asked for an exact 40-character summary, flash returned an empty body while eco succeeded.

Images work even though the catalog says otherwise

/v1/models reports eco as text-only. It still read images correctly.

ImageExpectedflasheco
Invoice with a total line49500OKOK
Bar chart, largest monthAprOKOK
Code screenshot, two output lines4,7OKOK
Typo spotting, differing line number3OKOK
Small text, last of 40 lines1273NGNG

Seven of eight, identical scores. Do not trust the catalog metadata blindly, but avoid depending on undocumented capability.

Three quirks specific to eco

Quality matches, but behavior differs in ways that matter operationally.

It rejects `reasoning_effort=ultra`

flash accepts it, eco returns HTTP 400.

{"error":{"code":"upstream_error","message":"The request could not be completed."}}

It is nondeterministic at `temperature=0`

Five identical requests produced one unique output on flash and three on eco. Do not rely on eco for reproducible tests or caching.

Thinking runs away on long summarization

With reasoning_effort=high and max_tokens=8000, the body came back empty. Both models spent the entire budget on reasoning. eco made me wait 82 seconds versus 45 for flash.

The workaround is a larger cap. At 32000, eco finished within 10,113 reasoning tokens.

One point I could not verify

The claim that both models use the same weights could not be confirmed directly.

  • system_fingerprint came back empty for both, five runs each
  • logprobs returned null, and top_logprobs was rejected with HTTP 400

The model field and the routing provider do differ, so the two are distinguishable. Whether the weights match is not.

Summary

  • The no-trade-off claim holds, with 18/18 on executable code tests for both
  • The delay is uneven: no gap on short turns, 2x on generation, 6x on long inputs
  • The upstreams differ: Volcengine versus BytePlus. This is not a quantization variant
  • eco has its own quirks: no ultra effort, nondeterministic at temp 0, runaway thinking
  • The split is clear: short conversations on eco, long documents and code editing on flash

コメント

タイトルとURLをコピーしました