Gemma 4 vs Qwen3
on Global Restaurant Chain

At a Glance

Both models identified real structural conditions in the same organization, but they stressed different vertices and told different stories about why. That divergence is itself a finding.

Speed

Gemma 4 was consistently faster across every pipeline phase except synthesis. On a DGX Spark running Ollama, it processed the same 709 documents 14% faster end-to-end.

Gemma 4's speed advantage comes from extraction — the longest phase. It processes documents faster per-token despite generating slightly more total tokens. Qwen3 was faster only at synthesis, where it produced a more compressed case summary.

Extraction Quality

Both models extracted the same number of claims (23) from the 7 center documents. The divergence is in edge extraction and classification discipline.

Qwen3 extracted 35% more observations, particularly from customer reviews (985 vs 782) and social media (947 vs 657). But it also produced 72 segments it couldn't classify — all in social media and news. Gemma 4 had zero classification failures across the entire corpus.

More extraction isn't necessarily better. The question is whether the additional volume improves diagnostic signal or adds noise. In this case, Qwen3's extra observations did not produce more sustained findings or higher scores. The additional volume made the corpus noisier without sharpening the diagnosis.

Source Breakdown

Scoring

Both runs used the finding_derived_v1 scoring mode with agent-powered vertex assessment and adversarial debate. Same caps were applied to both — limited center source types, no job postings, no employee reviews in the edge corpus.

The 0.066 gap in overall score is meaningful at this scale. But the more interesting divergence is in what each model identified as the primary stress point. Gemma 4 saw Truth under omission — the center narrative talks digital innovation but ignores labor conditions. Qwen3 saw Authority under misalignment — no single point of accountability for app or customer support.

Both reads are defensible. The same organization can have both conditions simultaneously. The question for pipeline development is whether the model should converge on the same diagnosis, or whether model-level divergence itself reveals complementary structural signals.

Skeptic Debate

Both models proposed 5 initial findings and sustained 2 through 2 rounds of adversarial debate. Identical throughput: 40%.

Gemma 4 pre-filtered 1 finding before debate, catching a weak candidate early. Qwen3 sent all 5 to the full debate process and used 56% more tokens to reach the same conclusion. In adversarial reasoning, Gemma 4 is more efficient — tighter arguments, less verbosity.

Diagnostic Depth

Sustained Findings

Gemma 4 found both an alignment (what works) and an omission (what's hidden). The labor finding — frontline employees overworked without recognition — is a deeper structural insight that Qwen3 missed entirely. Qwen3's authority finding about diffused ownership is valid but more surface-level.

Field Notes

Gemma 4 generated 8 field notes to Qwen3's 6. The additional signals Gemma 4 surfaced:

Verdict

The one thing Qwen3 did differently — not better, differently — is that it stressed Authority where Gemma 4 stressed Truth. In a production context, running both models on the same entity and comparing vertex stress could surface complementary conditions. But if you're picking one model for fleet runs, Gemma 4 wins on every axis that matters for pipeline reliability.

Methodology