local-ai-bench

Qwen3.8-Flash-Next and GLM-5.3-Flash on Ryzen 9 5950X · 128 GB DDR4-3200 · 3× RTX 3090

Current answer · as of 2026-10-12

For coding agents on one RTX 3090, run Qwen3.8-Flash-Next on the Strata engine.

Tested on one active RTX 3090 at PCIe 4.0 x8 in a 3-card AM4 host (Ryzen 9 5950X, 128 GB DDR4). Pick IQ3_S for task success (48 of 50 fresh issues solved, against 39 for Qwen3.8-27B on the registry's own 208k recipe; slowest minute of agentic decode 87 tok/s) or Q2_0 for speed (slowest minute 123 tok/s; 46 of 50 solved). Both recipes are submitted to the local-ai registry with passing gates. Avoid the OrcaRouter Uncensored fine-tune for coding (14 of 50). GLM-5.3-Flash stays under 15 tok/s on every route we tried on this box.

Agentic speed

123 tok/s

Slowest 60-second window of decode during a 10-minute coding-agent run.

Qwen3.8-Flash-Next · Strata · GGUF Q2_0 · 1 active RTX 3090, PCIe 4.0 x8 · 131k context

Limit: Single run per quant; Strata serves one request at a time. The previous best on this card (trellis/SGLang) was 40.9.

Evidence →

Fresh-issue coding

48 / 50 vs 39 / 50

Qwen3.8-Flash-Next (Strata IQ3_S) vs Qwen3.8-27B on the registry's 208k RTX 3090 recipe, on 50 private bug fixes opened after both models were released. Predeclared comparison (27B on our 131k launch): 48 vs 36, p = 0.0018.

One pass per task, same agent, sampling and 60k-token budget; 208k comparison p = 0.012 (exploratory)

430 attempts → 50 tasks · paired 13–1 · exact McNemar p = 0.0018 · commitment 5371c3b8d1ca…

Limit: On this 50-task set only; tasks skew easy; curated by Claude Opus.

Evidence →

GLM-5.3-Flash feasibility

< 15 tok/s

Best single-card decode across every tested route on this host: GGUF engines, exllamav3 exact mode, and Sero's NVMe-tier recipe (11.3 tok/s at a 90 GiB RAM cap).

GLM-5.3-Flash (321B total, 18B active) · 1 active RTX 3090 · 128 GB DDR4 · one NVMe at PCIe 4.0 x2

Limit: Hardware-bound here: NVMe first, then CPU and memory bandwidth (probe).

Evidence →

How we got here

  1. 10 / 03
    Fix

    Found and isolated a faulty CPU core

    Machine-check errors traced to core 2; it is taken offline at boot and CPU boost is disabled, leaving 15 cores at 3.4 GHz for every run that follows.

    decision 0004 decision 0010 Finding →

  2. 10 / 04
    Result

    Card order matters: +43–51 % prefill

    Putting the card on the CPU's PCIe lanes (GPU2, 13.5 GB/s) first instead of the chipset-attached card (6 GB/s) raised prefill by 43–51 % on multi-card runs.

    decision 0006 Finding →

  3. 10 / 04
    Negative result

    Flash-Next on trellis/SGLang: 55 tok/s in the lab, ~40 under a coding agent

    Sero's PR #144 launch reaches 55–57 tok/s lab decode and ~2,400 tok/s prefill, but the slowest minute of an agentic run stays near 40 tok/s on a PCIe x8 card. No tested x8 setting reached the 50 tok/s floor.

    decision 0011 Finding →

  4. 10 / 05
    Negative result

    GLM-5.3-Flash: under 15 tok/s on every route

    GGUF quants on llama.cpp and ik_llama (1–3 cards) and exllamav3 exact mode all stayed below the 15 tok/s gate.

    decision 0012 Finding →

  5. 10 / 05
    Fix

    PR #144 image ran out of VRAM in 2 of 4 runs; mitigated

    The image runs a 24 GB card down to 2 MiB free. Chunked prefill 8192 left 1,060 MiB free and ran 3 of 3 full sequences without a crash, decode unchanged, prefill −8 to −12 %.

    decision 0013 Finding →

  6. 10 / 06
    Result

    Public coding benchmark turned out to be memorised

    Flash-Next solved 30 of 30 SWE-bench-Lite tasks against 18 for Qwen3.8-27B, but its patches reproduce the original fixes almost word for word (median 100 % of the fix's added lines): evidence consistent with memorisation, not skill.

    decision 0014 Finding →

  7. 10 / 08
    Contribution

    Reported upstream to Sero

    Posted the PR #144 rerun with the out-of-memory finding and its mitigation, and opened a recipe-kit submission with this box's x8 data point.

    decision 0015 Finding →

  8. 10 / 08
    Result

    Strata engine: 123 and 87 tok/s at the slowest minute

    The Strata C++ engine with ISTA-DASLab GGUF quants held 123 tok/s (Q2_0) and 87 tok/s (IQ3_S) through 10-minute agentic runs on the same x8 card, with lab prefill 2,174–2,664 tok/s.

    ticket T41 Finding →

  9. 10 / 08
    Result

    moetier: what limits GLM here

    Sero's NVMe-tier GLM recipe gave 6.1 tok/s at a 55 GiB RAM cap and 11.3 at 90 GiB. Probe sampling shows the NVMe drive saturated at 55 GiB and the CPU expert lane the limit at 90 GiB.

    ticket T42 Finding →

  10. 10 / 10
    Tool

    Built a private fresh-issues benchmark

    430 attempts on Python issues opened after 1 September 2026 gave 50 validated test tasks. Flash-Next on Strata IQ3_S solved 48, Qwen3.8-27B 36 (13 vs 1 discordant tasks, exact McNemar p = 0.0018, declared in advance).

    decision 0016 ticket T43 Finding →

  11. 10 / 11
    Negative result

    OrcaRouter's uncensored Flash-Next: same speed, 14 of 50 solved

    Run on Strata through its manual compatibility pack, the uncensored IQ3_XXS fine-tune matched IQ3_S on speed (86 tok/s floor) but solved 14 of our 50 fresh tasks, with 72 truncated replies and 14 malformed tool calls.

    ticket T46 Finding →

  12. 10 / 12
    Result

    27B on the registry's own recipe: 39 of 50

    Re-running Qwen3.8-27B with the registry's exact 208k RTX 3090 recipe gave 39 of 50 (our 131k launch: 36). Strata IQ3_S still leads 48 to 39 (10 tasks vs 1, p = 0.012).

    ticket T48 Finding →

  13. 10 / 12
    Contribution

    Strata recipes and moetier data sent upstream

    Opened a registry PR with Strata IQ3_S and Q2_0 recipes for the RTX 3090 (six gates pass, 75 and 99.7 tok/s), and a moetier PR with this box's hardware record, the two GLM NVMe runs and a simulator calibration point.

    ticket T47 ticket T42 Finding →

Contributions upstream

What more hardware would test

  • No cost: Move the model NVMe to a CPU-attached x4 slot. Simulated 10.8 tok/s instead of 7.0 at a 55 GiB cap; little effect at 90 GiB, where the CPU binds.
  • Medium: Workstation platform: PCIe 4.0 x16 per GPU, 8-channel memory (e.g. EPYC). trellis Flash-Next towards Sero's x16 measurement on the same image (kit C1 61 tok/s, lab 67.5); GLM 23–27 tok/s (simulated on his EPYC box).
  • Large: One 96 GB Blackwell card (RTX PRO 6000). GLM 36–40 tok/s simulated (a lower bound; the 3090's fixed per-token time was used). Flash-Next experts mostly resident.
  • Not recommended: A 4th RTX 3090. No gain for these engines.
Constraints and success criteria →

Next

  • A second fresh slice (October issues) and the first non-Python slice (TypeScript, Go, Rust).
  • Qwen 4: download, launch and screen on day one with the same suite.
  • Repeat the Strata speed runs (3 runs incl. a cold start) to qualify them under the site's stability rule.
  • Run the OrcaRouter GGUF on llama.cpp to tell whether the fine-tune or Strata's compatibility repack causes its failures.
  • Demos for Sero: a recorded Strata coding-agent run and a long-context codebase Q&A.

281 result records; latest run 2026-10-12 11:48 AEST. All data →

Findings

Which setup runs coding agents fastest?

Measured on one RTX 3090 at PCIe 4.0 x8 (GPU2). The number that matters for an agent is the floor: the slowest 60-second window of decode during a 10-minute agentic coding run, not the lab's first 30 seconds.

Agentic floor by setup

Qwen3.8-Flash-NextGLM-5.3-FlashQwen3.8-27B
050100150Flash-Next GGUF Q2_0 · stratastrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2: slowest 60 s window 123 tok/s, 116 tool calls12327B EXL3 3.00 bpw · sglang-exl3sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2: slowest 60 s window 111 tok/s, 137 tool calls111Flash-Next GGUF IQ3_S · stratastrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2: slowest 60 s window 87.3 tok/s, 77 tool calls87.3Flash-Next GGUF IQ3_XXS · stratastrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2: slowest 60 s window 86.0 tok/s, 50 tool calls86.0Flash-Next EXL3 3.05 bpw · trellis/SGLangsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2: slowest 60 s window 40.9 tok/s, 72 tool calls40.9Flash-Next GGUF IQ4_XS · llama.cpp · 3 cardsllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: slowest 60 s window 38.6 tok/s, 70 tool calls38.6Flash-Next GGUF IQ4_XS · llama.cppfn-llamacpp-iq4xs-128k-1x-fit-gpu2: slowest 60 s window 23.8 tok/s, 49 tool calls23.8GLM GGUF IQ2_M · ik_llama.cpp · 3 cardsik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: slowest 60 s window 13.3 tok/s, 0 tool calls13.3GLM GGUF IQ2_M · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012: slowest 60 s window 11.4 tok/s, 26 tool calls11.4GLM GGUF IQ3_XXS · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012: slowest 60 s window 9.86 tok/s, 22 tool calls9.86GLM GGUF IQ2_M · ik_llama.cppik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2: slowest 60 s window 9.46 tok/s, 0 tool calls9.46GLM GGUF IQ2_M · llama.cppglm-llamacpp-iq2m-128k-1x-gpu2: slowest 60 s window 8.91 tok/s, 24 tool calls8.91GLM GGUF Q2_K · llama.cppllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2: slowest 60 s window 8.80 tok/s, 19 tool calls8.80GLM GGUF IQ4_XS · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012: slowest 60 s window 7.81 tok/s, 25 tool calls7.81target 50tok/s, slowest 60-second window of a 10-minute agentic coding run
Table view
ConfigFloor tok/sRun
Flash-Next GGUF Q2_0 · strata123strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
27B EXL3 3.00 bpw · sglang-exl3111sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Flash-Next GGUF IQ3_S · strata87.3strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Flash-Next GGUF IQ3_XXS · strata86.0strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Flash-Next EXL3 3.05 bpw · trellis/SGLang40.9sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Flash-Next GGUF IQ4_XS · llama.cpp · 3 cards38.6llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Flash-Next GGUF IQ4_XS · llama.cpp23.8fn-llamacpp-iq4xs-128k-1x-fit-gpu2
GLM GGUF IQ2_M · ik_llama.cpp · 3 cards13.3ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
GLM GGUF IQ2_M · llama.cpp · 3 cards11.4llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
GLM GGUF IQ3_XXS · llama.cpp · 3 cards9.86llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
GLM GGUF IQ2_M · ik_llama.cpp9.46ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
GLM GGUF IQ2_M · llama.cpp8.91glm-llamacpp-iq2m-128k-1x-gpu2
GLM GGUF Q2_K · llama.cpp8.80llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
GLM GGUF IQ4_XS · llama.cpp · 3 cards7.81llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
GLM GGUF IQ3_XXS · llama.cpp7.65llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
GLM EXL3 3.05 bpw · exllamav33.36exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

RAM, CPU and VRAM per phase

CPU cores busy (of 15 cores / 30 threads online)

prefilldecode
05101520Flash-Next GGUF Q2_0 · strata · Prefill: 1.33 CPU cores busy1.33Flash-Next GGUF Q2_0 · strata · Decode: 15.2 CPU cores busy15.2Strata Q2_0Flash-Next GGUF IQ3_S · strata · Prefill: 1.29 CPU cores busy1.29Flash-Next GGUF IQ3_S · strata · Decode: 15.2 CPU cores busy15.2Strata IQ3_S27B EXL3 3.00 bpw · sglang-exl3 · Prefill: 1.10 CPU cores busy1.1027B EXL3 3.00 bpw · sglang-exl3 · Decode: 1.11 CPU cores busy1.1127B EXL3Flash-Next EXL3 3.05 bpw · trellis/SGLang · Prefill: 1.28 CPU cores busy1.28Flash-Next EXL3 3.05 bpw · trellis/SGLang · Decode: 1.26 CPU cores busy1.26Flash-Next trellisFlash-Next GGUF IQ3_XXS · strata · Prefill: 1.34 CPU cores busy1.34Flash-Next GGUF IQ3_XXS · strata · Decode: 15.2 CPU cores busy15.2Strata IQ3_XXScores busy (mean)

Memory during decode

VRAM (GiB)host RAM: model and buffers (GiB)
020406080Flash-Next GGUF Q2_0 · strata: VRAM 23.6 GiB23.6Flash-Next GGUF Q2_0 · strata: host RAM (model and buffers) 33.6 GiB33.6Strata Q2_0Flash-Next GGUF IQ3_S · strata: VRAM 23.7 GiB23.7Flash-Next GGUF IQ3_S · strata: host RAM (model and buffers) 48.8 GiB48.8Strata IQ3_S27B EXL3 3.00 bpw · sglang-exl3: VRAM 22.6 GiB22.627B EXL3 3.00 bpw · sglang-exl3: host RAM (model and buffers) 5.37 GiB5.3727B EXL3Flash-Next EXL3 3.05 bpw · trellis/SGLang: VRAM 22.9 GiB22.9Flash-Next EXL3 3.05 bpw · trellis/SGLang: host RAM (model and buffers) 57.4 GiB57.4Flash-Next trellisFlash-Next GGUF IQ3_XXS · strata: VRAM 23.7 GiB23.7Flash-Next GGUF IQ3_XXS · strata: host RAM (model and buffers) 51.8 GiB51.8Strata IQ3_XXSGiB
SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
Flash-Next GGUF Q2_0 · strataIdle (loaded)23.5 GiB33.2 GiB39.1 GiB0.1 (p95 0.27)0 %0 GB/s100 W–
Prefill23.6 GiB33.6 GiB39.1 GiB1.33 (p95 2.88)93.4 %8.42 GB/s348 W2,584 tok/s
Decode23.6 GiB33.6 GiB39.1 GiB15.2 (p95 15.6)99.6 %0.78 GB/s381 W123 tok/s
Flash-Next GGUF IQ3_S · strataIdle (loaded)23.6 GiB48.4 GiB55.2 GiB0.19 (p95 0.29)0 %0 GB/s109 W–
Prefill23.6 GiB48.9 GiB55.2 GiB1.29 (p95 2.60)95.6 %11.6 GB/s322 W1,964 tok/s
Decode23.7 GiB48.8 GiB55.2 GiB15.2 (p95 15.6)99.6 %3.47 GB/s359 W93.8 tok/s
27B EXL3 3.00 bpw · sglang-exl3Idle (loaded)22.3 GiB5.44 GiB17.1 GiB1.10 (p95 1.38)0 %0 GB/s113 W–
Prefill22.6 GiB5.37 GiB17.1 GiB1.10 (p95 1.27)98.8 %0.18 GB/s384 W984 tok/s
Decode22.6 GiB5.37 GiB17.1 GiB1.11 (p95 1.28)95.3 %2.06 GB/s388 W108 tok/s
Flash-Next EXL3 3.05 bpw · trellis/SGLangIdle (loaded)21.1 GiB57.2 GiB54.8 GiB1.20 (p95 2.10)1.30 %0.05 GB/s152 W–
Prefill22.9 GiB57.5 GiB55.5 GiB1.28 (p95 1.81)74.5 %9.19 GB/s279 W2,131 tok/s
Decode22.9 GiB57.4 GiB55.8 GiB1.26 (p95 1.61)99.6 %6.38 GB/s309 W53.0 tok/s
Flash-Next GGUF IQ3_XXS · strataIdle (loaded)23.7 GiB51.8 GiB57.2 GiB0.09 (p95 0.28)0 %0 GB/s120 W–
Prefill23.7 GiB51.9 GiB57.2 GiB1.34 (p95 2.58)95.3 %12.5 GB/s320 W1,911 tok/s
Decode23.7 GiB51.8 GiB57.2 GiB15.2 (p95 15.6)99.9 %3.81 GB/s352 W92.8 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

Decode vs occupied context

Qwen3.8-Flash-NextGLM-5.3-FlashQwen3.8-27Blab C1 (first 30 s)stability: min 60 s window
050100150200050100150200occupied context (k tokens)decode tok/sFlash-Next target 50GLM gate 15128kfn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 49.3 tok/s at 166,667 occupied · passfn-trellis-3.05-0xsero-main-x8-gpu2 · stability min 60 s window 38.1 tok/s at 30,034 occupied · failfn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · passfn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.5 tok/s at 106,294 occupied · passfn-llamacpp-iq4xs-128k-1x-fit-gpu2 · stability min 60 s window 23.8 tok/s at 16,819 occupied · failfn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.1 tok/s at 106,294 occupied · passglm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failglm-llamacpp-iq2m-128k-1x-gpu2 · stability min 60 s window 8.91 tok/s at 9,047 occupied · failglm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.20 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · stability min 60 s window 8.80 tok/s at 6,658 occupied · failllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.30 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 10.0 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · stability min 60 s window 9.46 tok/s at 1,874 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 · lab C1 decode 8.20 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.30 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · stability min 60 s window 7.65 tok/s at 4,875 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.40 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · stability min 60 s window 8.85 tok/s at 9,732 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 · lab C1 decode 8.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · lab C1 decode 12.3 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · stability min 60 s window 11.4 tok/s at 8,118 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.5 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · stability min 60 s window 9.86 tok/s at 5,309 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.4 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 · lab C1 decode 12.2 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 11.0 tok/s at 6,174 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · stability min 60 s window 8.85 tok/s at 6,632 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.00 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 7.81 tok/s at 6,852 occupied · failllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.30 tok/s at 93,437 occupied · failllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 56.9 tok/s at 106,294 occupied · passllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · stability min 60 s window 38.6 tok/s at 39,226 occupied · failllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 57.1 tok/s at 106,294 occupied · passllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 · lab C1 decode 9.10 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 · lab C1 decode 11.2 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 14.0 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · stability min 60 s window 13.3 tok/s at 1,874 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 13.4 tok/s at 93,437 occupied · failllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 · lab C1 decode 44.3 tok/s at 106,294 occupied · passllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 · lab C1 decode 35.0 tok/s at 106,294 occupied · passik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 · lab C1 decode 10.2 tok/s at 93,437 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · stability min 60 s window 37.9 tok/s at 36,327 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 54.5 tok/s at 166,667 occupied · passexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.40 tok/s at 93,437 occupied · failexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · stability min 60 s window 3.36 tok/s at 4,589 occupied · failexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.50 tok/s at 93,437 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 55.8 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · stability min 60 s window 39.7 tok/s at 25,309 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 57.4 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · stability min 60 s window 40.1 tok/s at 29,027 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 53.7 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · stability min 60 s window 40.9 tok/s at 39,281 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 51.3 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 48.1 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · stability min 60 s window 37.4 tok/s at 34,417 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 49.7 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.9 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.5 tok/s at 25,179 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 49.5 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 49.9 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 53.5 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 38.8 tok/s at 28,209 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 56.4 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 55.0 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 40.4 tok/s at 30,539 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 45.0 tok/s at 166,667 occupied · passsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 · stability min 60 s window 111 tok/s at 52,045 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 48.8 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.8 tok/s at 28,542 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · passsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 · lab C1 decode 144 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 123 tok/s at 38,712 occupied · passstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · passglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 · lab C1 decode 6.10 tok/s at 93,437 occupied · failglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 · lab C1 decode 11.3 tok/s at 93,437 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · stability min 60 s window 39.0 tok/s at 28,256 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 56.1 tok/s at 166,667 occupied · passstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 78.1 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 87.3 tok/s at 39,624 occupied · passstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 75.5 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 · lab C1 decode 93.2 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 76.4 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 86.0 tok/s at 56,311 occupied · failstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 73.6 tok/s at 106,294 occupied · passsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 126 tok/s at 173,372 occupied · passsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · stability min 60 s window 90.1 tok/s at 68,013 occupied · passsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 139 tok/s at 173,372 occupied · pass
Table view
ModelConfigMeasureOccupiedtok/sOutcome
Qwen3.8-Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2lab C1 decode166,66749.3✓ pass
Qwen3.8-Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2stability min 60 s window30,03438.1! fail
Qwen3.8-Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2lab C1 decode166,66755.3✓ pass
Qwen3.8-Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2lab C1 decode106,29426.5✓ pass
Qwen3.8-Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2stability min 60 s window16,81923.8! fail
Qwen3.8-Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2lab C1 decode106,29426.1✓ pass
GLM-5.3-Flashglm-llamacpp-iq2m-128k-1x-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashglm-llamacpp-iq2m-128k-1x-gpu2stability min 60 s window9,0478.91! fail
GLM-5.3-Flashglm-llamacpp-iq2m-128k-1x-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab C1 decode93,4379.20! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2stability min 60 s window6,6588.80! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab C1 decode93,4379.30! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab C1 decode93,43710.0! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2stability min 60 s window1,8749.46! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2lab C1 decode93,4378.20! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab C1 decode93,4378.30! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2stability min 60 s window4,8757.65! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab C1 decode93,4378.40! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2stability min 60 s window9,7328.85! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2lab C1 decode93,4378.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012lab C1 decode93,43712.3! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012stability min 60 s window8,11811.4! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab C1 decode93,43710.5! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012stability min 60 s window5,3099.86! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab C1 decode93,43710.4! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012lab C1 decode93,43712.2! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,43711.8! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012stability min 60 s window6,17411.0! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,43711.8! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2stability min 60 s window6,6328.85! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,4378.00! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012stability min 60 s window6,8527.81! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,4378.30! fail
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012lab C1 decode106,29456.9✓ pass
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012stability min 60 s window39,22638.6! fail
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012lab C1 decode106,29457.1✓ pass
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2lab C1 decode93,4379.10! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012lab C1 decode93,43711.2! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab C1 decode93,43714.0! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012stability min 60 s window1,87413.3! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab C1 decode93,43713.4! fail
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012lab C1 decode106,29444.3✓ pass
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012lab C1 decode106,29435.0✓ pass
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012lab C1 decode93,43710.2! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2lab C1 decode166,66755.6✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2stability min 60 s window36,32737.9! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2lab C1 decode166,66754.5✓ pass
GLM-5.3-Flashexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab C1 decode93,4377.40! fail
GLM-5.3-Flashexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2stability min 60 s window4,5893.36! fail
GLM-5.3-Flashexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab C1 decode93,4377.50! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab C1 decode166,66755.8✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2stability min 60 s window25,30939.7! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab C1 decode166,66757.4✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2lab C1 decode166,66755.6✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2stability min 60 s window29,02740.1! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2lab C1 decode166,66754.6✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2lab C1 decode166,66753.7✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2stability min 60 s window39,28140.9! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2lab C1 decode166,66751.3✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2lab C1 decode166,66748.1✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2stability min 60 s window34,41737.4! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2lab C1 decode166,66749.7✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66754.9✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window25,17939.5! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66749.5✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab C1 decode166,66749.9✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66753.5✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window28,20938.8! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66756.4✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66755.0✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window30,53940.4! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66745.0✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2stability min 60 s window52,045111! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66748.8✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window28,54239.8! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66754.6✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2lab C1 decode106,294144✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2lab C1 decode106,294100✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2stability min 60 s window38,712123✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2lab C1 decode106,294100✓ pass
GLM-5.3-Flashglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2lab C1 decode93,4376.10! fail
GLM-5.3-Flashglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2lab C1 decode93,43711.3! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2lab C1 decode166,66755.3✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2stability min 60 s window28,25639.0! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2lab C1 decode166,66756.1✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2lab C1 decode106,29478.1✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2stability min 60 s window39,62487.3✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2lab C1 decode106,29475.5✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2lab C1 decode106,29493.2✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2lab C1 decode106,29476.4✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2stability min 60 s window56,31186.0! fail
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2lab C1 decode106,29473.6✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab C1 decode173,372126✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2stability min 60 s window68,01390.1✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab C1 decode173,372139✓ pass

Findings

How often do the tested setups solve fresh issues?

Public coding benchmarks are older than these models' training data. On SWE-bench-Lite, Flash-Next reproduces the original fixes almost verbatim, so we built a private set of issues opened after both models were released.

Fifty real bug fixes from open-source Python projects, where both the GitHub issue and the pull request that fixed it were created on or after 1 September 2026. Both Qwen3.8 models were released before that (27B on 5 August, Flash-Next on 24 August), so neither could have seen these fixes in training. The tasks stay private; only scores and a commitment hash are published.

Test tasks
50
Repositories
47
Created
09-01 → 10-06 2026
Validated from
430 attempts
Median fix
7 lines added

Solved tasks

One uncapped pass per task, same harness and sampling for every model. Whiskers: 95 % Wilson interval (±14 points at n = 50).

Strata · Flash-Next IQ3_SStrata · Flash-Next Q2_0Qwen3.8-27B · registry 208k recipeQwen3.8-27B · EXL3 3.00 bpw · 131k launchStrata · OrcaRouter Uncensored IQ3_XXS
0%20%40%60%80%100%Strata · Flash-Next IQ3_S: 48/50 solved · 95 % CI 87%–99% · 83.0 tok/s48/50Strata IQ3_SStrata · Flash-Next Q2_0: 46/50 solved · 95 % CI 81%–97% · 123 tok/s46/50Strata Q2_0Qwen3.8-27B · registry 208k recipe: 39/50 solved · 95 % CI 65%–87% · 112 tok/s39/5027B (208k recipe)Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 36/50 solved · 95 % CI 58%–83% · 119 tok/s36/5027B (131k)Strata · OrcaRouter Uncensored IQ3_XXS: 14/50 solved · 95 % CI 17%–42% · 85.2 tok/s14/50Orca Uncensoredsolved
Table view
ConfigSolved95 % CIDecode tok/sTool callsParse failuresTruncationsRepetition hits
Strata · Flash-Next IQ3_S48/5087%–99%83.03,366116
Strata · Flash-Next Q2_046/5081%–97%1232,748037
Qwen3.8-27B · registry 208k recipe39/5065%–87%1122,59812743
Qwen3.8-27B · EXL3 3.00 bpw · 131k launch36/5058%–83%1192,68243967
Strata · OrcaRouter Uncensored IQ3_XXS14/5017%–42%85.23,084147214

Primary comparison, decided before any run

Strata · Flash-Next IQ3_S vs Qwen3.8-27B · EXL3 3.00 bpw · 131k launch on the same 50 tasks. Exact McNemar test on the tasks where exactly one of them succeeded.

35Both solved
13Only Strata IQ3_S
1Only 27B (131k)
1Neither

p = 0.0018 (13 vs 1 discordant tasks; holds after Bonferroni correction for 3 comparisons)

Exploratory: Strata · Flash-Next IQ3_S vs Strata · Flash-Next Q2_0 3–1, p = 0.625; Strata · Flash-Next IQ3_S vs Qwen3.8-27B · registry 208k recipe 10–1, p = 0.012; Strata · Flash-Next IQ3_S vs Strata · OrcaRouter Uncensored IQ3_XXS 35–1, p = 0.000.

By difficulty

Difficulty from the size of the upstream fix: easy = 1 file, ≤ 10 lines; medium = 1–2 files, ≤ 50 lines; hard = 3+ files or more than 50 lines. Bar labels are solved counts.

Strata · Flash-Next IQ3_SStrata · Flash-Next Q2_0Qwen3.8-27B · registry 208k recipeQwen3.8-27B · EXL3 3.00 bpw · 131k launchStrata · OrcaRouter Uncensored IQ3_XXS
0%20%40%60%80%100%Strata · Flash-Next IQ3_S: 23/24 easy23Strata · Flash-Next Q2_0: 23/24 easy23Qwen3.8-27B · registry 208k recipe: 21/24 easy21Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 19/24 easy19Strata · OrcaRouter Uncensored IQ3_XXS: 7/24 easy7easy (24)Strata · Flash-Next IQ3_S: 19/20 medium19Strata · Flash-Next Q2_0: 18/20 medium18Qwen3.8-27B · registry 208k recipe: 14/20 medium14Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 14/20 medium14Strata · OrcaRouter Uncensored IQ3_XXS: 4/20 medium4medium (20)Strata · Flash-Next IQ3_S: 6/6 hard6Strata · Flash-Next Q2_0: 5/6 hard5Qwen3.8-27B · registry 208k recipe: 4/6 hard4Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 3/6 hard3Strata · OrcaRouter Uncensored IQ3_XXS: 3/6 hard3hard (6)share solved

Why a fresh set: memorised fixes on the public benchmark

Share of the upstream fix's added lines that the model's patch reproduces word for word (median over tasks). On 30 public SWE-bench-Lite tasks Flash-Next, at every quantisation, rewrites the original fix almost verbatim. On the fresh slice it writes its own.

SWE-bench-Lite (public)fresh · Strata · Flash-Next IQ3_Sfresh · Strata · Flash-Next Q2_0fresh · Qwen3.8-27B · EXL3 3.00 bpw · 131k launchfresh · Strata · OrcaRouter Uncensored IQ3_XXS
0%20%40%60%80%100%SWE-bench-Lite (public, 2023): median 100% of the upstream fix reproduced verbatim100%Fresh slice: median 20% reproduced verbatim20%Strata IQ3_SSWE-bench-Lite (public, 2023): median 100% of the upstream fix reproduced verbatim100%Fresh slice: median 17% reproduced verbatim17%Strata Q2_0SWE-bench-Lite (public, 2023): median 14% of the upstream fix reproduced verbatim14%Fresh slice: median 17% reproduced verbatim17%27B (131k)SWE-bench-Lite (public, 2023): median 100% of the upstream fix reproduced verbatim100%Fresh slice: median 10% reproduced verbatim10%Orca Uncensoredmedian verbatim share of the upstream fix
Table view
ConfigLite overlapLite solvedFresh overlapFresh solved
Strata · Flash-Next IQ3_S100% (18/30 ≥ 90 %)29/3020% (3/50 ≥ 90 %)48/50
Strata · Flash-Next Q2_0100% (18/30 ≥ 90 %)28/3017% (2/50 ≥ 90 %)46/50
Qwen3.8-27B · EXL3 3.00 bpw · 131k launch14% (4/27 ≥ 90 %)18/3017% (4/50 ≥ 90 %)36/50
Strata · OrcaRouter Uncensored IQ3_XXS100% (17/21 ≥ 90 %)21/3010% (3/50 ≥ 90 %)14/50

Not completed: Flash-Next · trellis EXL3 3.05 bpw: stopped by the operator at 9 of 50 tasks (same weights as Strata, ~43 tok/s; not expected to differ measurably at n = 50).

Older evidence: public SWE-bench-Lite comparison (memorised; superseded).

Findings

Is GLM-5.3-Flash practical on this machine?

No: the best route on one RTX 3090 here is 11.3 tok/s decode. GLM-5.3-Flash has 321B parameters (18B active, 288 experts per layer), so even at ~3 bits most of it lives in RAM or on disk.

GLM-5.3-Flash: best lab decode per route

05101520GGUF IQ2_M · ik_llama.cpp · 3 cardsik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: lab C1 decode 14.0 tok/s14.0GGUF IQ2_M · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012: lab C1 decode 12.3 tok/s12.3EXL3 3.05 bpw · NVMe tier 90 GiBglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2: lab C1 decode 11.3 tok/s11.3GGUF IQ3_XXS · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012: lab C1 decode 10.5 tok/s10.5GGUF IQ2_M · ik_llama.cppik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2: lab C1 decode 10.0 tok/s10.0GGUF IQ2_M · llama.cppglm-llamacpp-iq2m-128k-1x-gpu2: lab C1 decode 9.60 tok/s9.60GGUF Q2_K · llama.cppllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2: lab C1 decode 9.30 tok/s9.30GGUF IQ3_XXS · llama.cppllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2: lab C1 decode 8.40 tok/s8.40GGUF IQ4_XS · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012: lab C1 decode 8.30 tok/s8.30EXL3 3.05 bpw · exllamav3exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2: lab C1 decode 7.50 tok/s7.50EXL3 3.05 bpw · NVMe tier 55 GiBglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2: lab C1 decode 6.10 tok/s6.10gate 15lab C1 decode, tok/s

What limits each setup (probe)

GPU busyPCIe host→GPU, % of measured 13.5 GB/sNVMe read, % of 3.36 GB/sCPU busy
0%20%40%60%80%100%GLM-5.3 · NVMe recipe · 55 GiB RAM: GPU busy 87 % of samples with a kernel running87%GLM-5.3 · NVMe recipe · 55 GiB RAM: PCIe in 3.34 GB/s of 13.5 (p95 6.17)25%GLM-5.3 · NVMe recipe · 55 GiB RAM: NVMe read 3.08 GB/s of 3.3692%GLM-5.3 · NVMe recipe · 55 GiB RAM: CPU 62% busy (expert-compute cores)62%GLM-5.3 55 GiBGLM-5.3 · NVMe recipe · 90 GiB RAM: GPU busy 96 % of samples with a kernel running96%GLM-5.3 · NVMe recipe · 90 GiB RAM: PCIe in 6.42 GB/s of 13.5 (p95 14.27)48%GLM-5.3 · NVMe recipe · 90 GiB RAM: NVMe read 1.31 GB/s of 3.3639%GLM-5.3 · NVMe recipe · 90 GiB RAM: CPU 83% busy (expert-compute cores)83%GLM-5.3 90 GiBFlash-Next · trellis · agentic run: GPU busy 89 % of samples with a kernel running89%Flash-Next · trellis · agentic run: PCIe in 7.77 GB/s of 13.5 (p95 14.25)58%Flash-Next · trellis · agentic run: NVMe read 0.0 GB/s of 3.360%Flash-Next · trellis · agentic run: CPU 4% busy (all cores)4%Flash-Next agenticshare of ceiling
Table view
ArmPhaseGPUPCIeNVMeCPU
GLM-5.3 · NVMe recipe · 55 GiB RAMkit87 % of samples with a kernel running3.34 GB/s of 13.5 (p95 6.17)3.08 GB/s of 3.3662% busy (expert-compute cores)
GLM-5.3 · NVMe recipe · 90 GiB RAMkit96 % of samples with a kernel running6.42 GB/s of 13.5 (p95 14.27)1.31 GB/s of 3.3683% busy (expert-compute cores)
Flash-Next · trellis · agentic runstab89 % of samples with a kernel running7.77 GB/s of 13.5 (p95 14.25)0.0 GB/s of 3.364% busy (all cores)

Findings

Which hardware limits affect performance?

The same card behaves differently depending on where it sits and what feeds it.

RAM, CPU and VRAM per phase

CPU cores busy (of 15 cores / 30 threads online)

prefilldecode
05101520Flash-Next GGUF Q2_0 · strata · Prefill: 1.33 CPU cores busy1.33Flash-Next GGUF Q2_0 · strata · Decode: 15.2 CPU cores busy15.2Strata Q2_0Flash-Next GGUF IQ3_S · strata · Prefill: 1.29 CPU cores busy1.29Flash-Next GGUF IQ3_S · strata · Decode: 15.2 CPU cores busy15.2Strata IQ3_S27B EXL3 3.00 bpw · sglang-exl3 · Prefill: 1.10 CPU cores busy1.1027B EXL3 3.00 bpw · sglang-exl3 · Decode: 1.11 CPU cores busy1.1127B EXL3Flash-Next EXL3 3.05 bpw · trellis/SGLang · Prefill: 1.28 CPU cores busy1.28Flash-Next EXL3 3.05 bpw · trellis/SGLang · Decode: 1.26 CPU cores busy1.26Flash-Next trellisFlash-Next GGUF IQ3_XXS · strata · Prefill: 1.34 CPU cores busy1.34Flash-Next GGUF IQ3_XXS · strata · Decode: 15.2 CPU cores busy15.2Strata IQ3_XXScores busy (mean)

Memory during decode

VRAM (GiB)host RAM: model and buffers (GiB)
020406080Flash-Next GGUF Q2_0 · strata: VRAM 23.6 GiB23.6Flash-Next GGUF Q2_0 · strata: host RAM (model and buffers) 33.6 GiB33.6Strata Q2_0Flash-Next GGUF IQ3_S · strata: VRAM 23.7 GiB23.7Flash-Next GGUF IQ3_S · strata: host RAM (model and buffers) 48.8 GiB48.8Strata IQ3_S27B EXL3 3.00 bpw · sglang-exl3: VRAM 22.6 GiB22.627B EXL3 3.00 bpw · sglang-exl3: host RAM (model and buffers) 5.37 GiB5.3727B EXL3Flash-Next EXL3 3.05 bpw · trellis/SGLang: VRAM 22.9 GiB22.9Flash-Next EXL3 3.05 bpw · trellis/SGLang: host RAM (model and buffers) 57.4 GiB57.4Flash-Next trellisFlash-Next GGUF IQ3_XXS · strata: VRAM 23.7 GiB23.7Flash-Next GGUF IQ3_XXS · strata: host RAM (model and buffers) 51.8 GiB51.8Strata IQ3_XXSGiB
SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
Flash-Next GGUF Q2_0 · strataIdle (loaded)23.5 GiB33.2 GiB39.1 GiB0.1 (p95 0.27)0 %0 GB/s100 W–
Prefill23.6 GiB33.6 GiB39.1 GiB1.33 (p95 2.88)93.4 %8.42 GB/s348 W2,584 tok/s
Decode23.6 GiB33.6 GiB39.1 GiB15.2 (p95 15.6)99.6 %0.78 GB/s381 W123 tok/s
Flash-Next GGUF IQ3_S · strataIdle (loaded)23.6 GiB48.4 GiB55.2 GiB0.19 (p95 0.29)0 %0 GB/s109 W–
Prefill23.6 GiB48.9 GiB55.2 GiB1.29 (p95 2.60)95.6 %11.6 GB/s322 W1,964 tok/s
Decode23.7 GiB48.8 GiB55.2 GiB15.2 (p95 15.6)99.6 %3.47 GB/s359 W93.8 tok/s
27B EXL3 3.00 bpw · sglang-exl3Idle (loaded)22.3 GiB5.44 GiB17.1 GiB1.10 (p95 1.38)0 %0 GB/s113 W–
Prefill22.6 GiB5.37 GiB17.1 GiB1.10 (p95 1.27)98.8 %0.18 GB/s384 W984 tok/s
Decode22.6 GiB5.37 GiB17.1 GiB1.11 (p95 1.28)95.3 %2.06 GB/s388 W108 tok/s
Flash-Next EXL3 3.05 bpw · trellis/SGLangIdle (loaded)21.1 GiB57.2 GiB54.8 GiB1.20 (p95 2.10)1.30 %0.05 GB/s152 W–
Prefill22.9 GiB57.5 GiB55.5 GiB1.28 (p95 1.81)74.5 %9.19 GB/s279 W2,131 tok/s
Decode22.9 GiB57.4 GiB55.8 GiB1.26 (p95 1.61)99.6 %6.38 GB/s309 W53.0 tok/s
Flash-Next GGUF IQ3_XXS · strataIdle (loaded)23.7 GiB51.8 GiB57.2 GiB0.09 (p95 0.28)0 %0 GB/s120 W–
Prefill23.7 GiB51.9 GiB57.2 GiB1.34 (p95 2.58)95.3 %12.5 GB/s320 W1,911 tok/s
Decode23.7 GiB51.8 GiB57.2 GiB15.2 (p95 15.6)99.9 %3.81 GB/s352 W92.8 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

What limits each setup (probe)

GPU busyPCIe host→GPU, % of measured 13.5 GB/sNVMe read, % of 3.36 GB/sCPU busy
0%20%40%60%80%100%GLM-5.3 · NVMe recipe · 55 GiB RAM: GPU busy 87 % of samples with a kernel running87%GLM-5.3 · NVMe recipe · 55 GiB RAM: PCIe in 3.34 GB/s of 13.5 (p95 6.17)25%GLM-5.3 · NVMe recipe · 55 GiB RAM: NVMe read 3.08 GB/s of 3.3692%GLM-5.3 · NVMe recipe · 55 GiB RAM: CPU 62% busy (expert-compute cores)62%GLM-5.3 55 GiBGLM-5.3 · NVMe recipe · 90 GiB RAM: GPU busy 96 % of samples with a kernel running96%GLM-5.3 · NVMe recipe · 90 GiB RAM: PCIe in 6.42 GB/s of 13.5 (p95 14.27)48%GLM-5.3 · NVMe recipe · 90 GiB RAM: NVMe read 1.31 GB/s of 3.3639%GLM-5.3 · NVMe recipe · 90 GiB RAM: CPU 83% busy (expert-compute cores)83%GLM-5.3 90 GiBFlash-Next · trellis · agentic run: GPU busy 89 % of samples with a kernel running89%Flash-Next · trellis · agentic run: PCIe in 7.77 GB/s of 13.5 (p95 14.25)58%Flash-Next · trellis · agentic run: NVMe read 0.0 GB/s of 3.360%Flash-Next · trellis · agentic run: CPU 4% busy (all cores)4%Flash-Next agenticshare of ceiling
Table view
ArmPhaseGPUPCIeNVMeCPU
GLM-5.3 · NVMe recipe · 55 GiB RAMkit87 % of samples with a kernel running3.34 GB/s of 13.5 (p95 6.17)3.08 GB/s of 3.3662% busy (expert-compute cores)
GLM-5.3 · NVMe recipe · 90 GiB RAMkit96 % of samples with a kernel running6.42 GB/s of 13.5 (p95 14.27)1.31 GB/s of 3.3683% busy (expert-compute cores)
Flash-Next · trellis · agentic runstab89 % of samples with a kernel running7.77 GB/s of 13.5 (p95 14.25)0.0 GB/s of 3.364% busy (all cores)

Measured host → device bandwidth per card

051015GPU0 · bus 00000000:04:00.0 · x820261002T205350Z-hw-bw-gpu0-current-hw_bw: 6.03 GB/s host to device6.03GPU1 · bus 00000000:0B:00.0 · x820261004T091142Z-hw-bw-gpu1-current-hw_bw: 13.47 GB/s host to device13.5GPU2 · bus 00000000:0C:00.0 · x820261002T205342Z-hw-bw-gpu2-current-hw_bw: 13.47 GB/s host to device13.5GB/s host → device

Recipes

Tested launch commands with the numbers they produced on this box. Commands are the exact ones the harness ran (GPU UUID and paths are this box's; adjust them). Measured on one RTX 3090 at PCIe 4.0 x8 unless noted.

Flash-Next · Strata · IQ3_S (best task success)

Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json

Lab decode
78.1 tok/s
Lab prefill
2,178 tok/s
Agentic floor
87.3 tok/s
Load time
43.0 s

Resources during execution

SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
Flash-Next GGUF IQ3_S · strataIdle (loaded)23.6 GiB48.4 GiB55.2 GiB0.19 (p95 0.29)0 %0 GB/s109 W–
Prefill23.6 GiB48.9 GiB55.2 GiB1.29 (p95 2.60)95.6 %11.6 GB/s322 W1,964 tok/s
Decode23.7 GiB48.8 GiB55.2 GiB15.2 (p95 15.6)99.6 %3.47 GB/s359 W93.8 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

Needs: 1× 24 GB NVIDIA GPU (driver ≥ 580), ~60 GB free RAM, 90 GB disk, Docker with the NVIDIA runtime.

One-time setup

git clone https://github.com/Niko1221/Strata && cd Strata && git checkout d5ea7133741e67743c0e886bb426c0ce8d69cf6c
docker build -t local-ai-bench/strata:v0.1.40.3-sm86 --build-arg CUDA_ARCHITECTURES=86 --build-arg BUILD_VISION=0 .
hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF --revision ed59f92082b1e93c0e96d60a8b11aab089b52f09 --include 'IQ3_S/*' --local-dir $MODELS/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Run

recorded by tools/run.py (artifacts/runs/20261011T191447Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172

Learnings

  • First start needs network: it builds a pack in /data and downloads ~5 GB of MTP draft-layer tensors from Qwen/Qwen3.8-Flash-Next (not revision-pinned). Later starts are offline.
  • Needs --ulimit memlock=-1 (pins 34–50 GB of experts).
  • Serves one request at a time; set "parallel" in /data/config for concurrency.
  • No logprobs on the CUDA server.
  • The first streamed chunk is an empty role delta: measure time-to-first-token on the first chunk with content.

Flash-Next · Strata · Q2_0 (fastest)

Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json

Lab decode
103 tok/s
Lab prefill
2,664 tok/s
Agentic floor
123 tok/s
Load time
25.1 s

Resources during execution

SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
Flash-Next GGUF Q2_0 · strataIdle (loaded)23.5 GiB33.2 GiB39.1 GiB0.1 (p95 0.27)0 %0 GB/s100 W–
Prefill23.6 GiB33.6 GiB39.1 GiB1.33 (p95 2.88)93.4 %8.42 GB/s348 W2,584 tok/s
Decode23.6 GiB33.6 GiB39.1 GiB15.2 (p95 15.6)99.6 %0.78 GB/s381 W123 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

Needs: 1× 24 GB NVIDIA GPU, ~45 GB free RAM, 72 GB disk.

One-time setup

Same image as IQ3_S; download 'Q2_0/*' instead. Shard 2 is byte-identical across quants and can be hardlinked.

Run

recorded by tools/run.py (artifacts/runs/20261012T014808Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172

Learnings

  • Same first-start and memlock notes as IQ3_S.

Qwen3.8-27B · registry 208k recipe (SGLang-EXL3, MTP, vision)

Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json

Lab decode
139 tok/s
Lab prefill
836 tok/s
Agentic floor
90.1 tok/s
Load time
113 s

Needs: 1× 24 GB NVIDIA GPU, ~8 GB RAM.

Run

recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt

Learnings

  • Byte-faithful copy of the local-ai registry's qwen3.8-27b.sglang.208k recipe for the RTX 3090 (image c8922bd7…, 212,992 context).
  • 39/50 on our fresh-issues benchmark; agentic floor 90 tok/s, lab decode 126–139 tok/s.

Qwen3.8-27B · SGLang-EXL3 · 3.00 bpw + MTP

Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json

Lab decode
144 tok/s
Lab prefill
740 tok/s
Load time
131 s

Resources during execution

SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
27B EXL3 3.00 bpw · sglang-exl3Idle (loaded)22.3 GiB5.44 GiB17.1 GiB1.10 (p95 1.38)0 %0 GB/s113 W–
Prefill22.6 GiB5.37 GiB17.1 GiB1.10 (p95 1.27)98.8 %0.18 GB/s384 W984 tok/s
Decode22.6 GiB5.37 GiB17.1 GiB1.11 (p95 1.28)95.3 %2.06 GB/s388 W108 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

Needs: 1× 24 GB NVIDIA GPU, ~8 GB RAM.

Run

recorded by tools/run.py (artifacts/runs/20261010T105104Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt

Learnings

  • The server advertises 204,800 context but the KV pool on 24 GB holds 135,424 tokens; this launch sets 131,072 so long prompts fail clearly instead of mid-generation.
  • At temperature 0.7 and 8,192-token replies it looped or truncated often in agentic runs (67 repetition hits on 50 tasks).

Flash-Next · trellis/SGLang · EXL3 3.05 bpw (Sero's registry route)

Qwen3.8-Flash-Next · launch launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json

Lab decode
56.4 tok/s
Lab prefill
2,240 tok/s
Agentic floor
40.4 tok/s
Load time
267 s

Resources during execution

SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
Flash-Next EXL3 3.05 bpw · trellis/SGLangIdle (loaded)21.1 GiB57.2 GiB54.8 GiB1.20 (p95 2.10)1.30 %0.05 GB/s152 W–
Prefill22.9 GiB57.5 GiB55.5 GiB1.28 (p95 1.81)74.5 %9.19 GB/s279 W2,131 tok/s
Decode22.9 GiB57.4 GiB55.8 GiB1.26 (p95 1.61)99.6 %6.38 GB/s309 W53.0 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

Needs: 1× 24 GB GPU on the widest PCIe link you have, 75 GB free RAM (page-locked), 86 GB on a local NVMe.

Run

recorded by tools/run.py (artifacts/runs/20261010T105600Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder

Learnings

  • Use --chunked-prefill-size 8192 on 24 GB cards (the 12288 default ran out of memory in 2 of 4 runs).
  • Keep the expert-cache reserve at 8.26 GB: Triton kernels load after the server reports ready.
  • Only 3.05 bpw works: the offload store is 3-bit only.
  • Load takes ~260 s; text and images supported.

GLM-5.3-Flash · glm53-flash-offload v4.1 · NVMe tier, 90 GiB (experimental)

GLM-5.3-Flash · launch launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json

Lab decode
11.3 tok/s
Lab prefill
630 tok/s
Load time
113 s

Needs: 1× 24 GB GPU, 90 GiB RAM for the container, 125 GB weights + 117 GB record store on NVMe.

One-time setup

mkdir -p $STORE && chattr +C $STORE   # btrfs: no copy-on-write for O_DIRECT
docker run --rm -v $MODELS/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v $STORE:/nvx ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 pack-store

Run

recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16

Learnings

  • 11 tok/s: usable for batch work, not interactive coding.
  • Pin the CPU layout explicitly when cores are offline (GLM53_MAIN_CPUS, GLM53_NV_*).
  • --memory and --memory-swap must be equal or pinned pages escape the cap.

Models

One card per model: what it is, which builds we ran, what we learned running it, and links to recipes and raw results.

Qwen3.8-Flash-Next

180B total: 125B language model (6B active; 512 experts, 10 routed + 1 shared per token) + 51B n-gram table + 4B MTP head

Best agentic floor here: 123 tok/s

Model card →

GLM-5.3-Flash

321B total, 18B active

Best agentic floor here: 13.3 tok/s

Model card →

Models

Model card

Qwen3.8-Flash-Next

huggingface.co/Qwen/Qwen3.8-Flash-Next · released 2026-08-24

Fresh-issues slice: Strata · Flash-Next IQ3_S 48/50; Strata · Flash-Next Q2_0 46/50; Strata · OrcaRouter Uncensored IQ3_XXS 14/50 details

Builds we ran

BuildWeightsEngineSize
EXL3 3.05 bpw (h5, ng5)turboderp/Qwen3.8-Flash-Next-exl3
@69e33439ae95
trellis/SGLang (Sero's image)85 GB
GSQ-RCO GGUF Q2_0ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
@ed59f92082b1
Strata66 GB
GSQ-RCO GGUF IQ3_SISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
@ed59f92082b1
Strata84 GB
GGUF IQ4_XS (bartowski)bartowski/Qwen3.8-Flash-Next-GGUFllama.cpp, 1–3 cards (experiment only)98 GB
OrcaRouter Uncensored IQ3_XXS (fine-tune)orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF
@e43d00f4e2b8
Strata (manual compatibility pack, docs/ORCA.md)85 GB
GSQ-RCO Coder IQ1_M (expert-pruned; stopped after lab and TTFT)ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF
@5348543e0147
Strata58 GB

Resources during execution

CPU cores busy (of 15 cores / 30 threads online)

prefilldecode
05101520Flash-Next GGUF Q2_0 · strata · Prefill: 1.33 CPU cores busy1.33Flash-Next GGUF Q2_0 · strata · Decode: 15.2 CPU cores busy15.2Strata Q2_0Flash-Next GGUF IQ3_S · strata · Prefill: 1.29 CPU cores busy1.29Flash-Next GGUF IQ3_S · strata · Decode: 15.2 CPU cores busy15.2Strata IQ3_SFlash-Next EXL3 3.05 bpw · trellis/SGLang · Prefill: 1.28 CPU cores busy1.28Flash-Next EXL3 3.05 bpw · trellis/SGLang · Decode: 1.26 CPU cores busy1.26Flash-Next trellisFlash-Next GGUF IQ3_XXS · strata · Prefill: 1.34 CPU cores busy1.34Flash-Next GGUF IQ3_XXS · strata · Decode: 15.2 CPU cores busy15.2Strata IQ3_XXScores busy (mean)

Memory during decode

VRAM (GiB)host RAM: model and buffers (GiB)
020406080Flash-Next GGUF Q2_0 · strata: VRAM 23.6 GiB23.6Flash-Next GGUF Q2_0 · strata: host RAM (model and buffers) 33.6 GiB33.6Strata Q2_0Flash-Next GGUF IQ3_S · strata: VRAM 23.7 GiB23.7Flash-Next GGUF IQ3_S · strata: host RAM (model and buffers) 48.8 GiB48.8Strata IQ3_SFlash-Next EXL3 3.05 bpw · trellis/SGLang: VRAM 22.9 GiB22.9Flash-Next EXL3 3.05 bpw · trellis/SGLang: host RAM (model and buffers) 57.4 GiB57.4Flash-Next trellisFlash-Next GGUF IQ3_XXS · strata: VRAM 23.7 GiB23.7Flash-Next GGUF IQ3_XXS · strata: host RAM (model and buffers) 51.8 GiB51.8Strata IQ3_XXSGiB
SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
Flash-Next GGUF Q2_0 · strataIdle (loaded)23.5 GiB33.2 GiB39.1 GiB0.1 (p95 0.27)0 %0 GB/s100 W–
Prefill23.6 GiB33.6 GiB39.1 GiB1.33 (p95 2.88)93.4 %8.42 GB/s348 W2,584 tok/s
Decode23.6 GiB33.6 GiB39.1 GiB15.2 (p95 15.6)99.6 %0.78 GB/s381 W123 tok/s
Flash-Next GGUF IQ3_S · strataIdle (loaded)23.6 GiB48.4 GiB55.2 GiB0.19 (p95 0.29)0 %0 GB/s109 W–
Prefill23.6 GiB48.9 GiB55.2 GiB1.29 (p95 2.60)95.6 %11.6 GB/s322 W1,964 tok/s
Decode23.7 GiB48.8 GiB55.2 GiB15.2 (p95 15.6)99.6 %3.47 GB/s359 W93.8 tok/s
Flash-Next EXL3 3.05 bpw · trellis/SGLangIdle (loaded)21.1 GiB57.2 GiB54.8 GiB1.20 (p95 2.10)1.30 %0.05 GB/s152 W–
Prefill22.9 GiB57.5 GiB55.5 GiB1.28 (p95 1.81)74.5 %9.19 GB/s279 W2,131 tok/s
Decode22.9 GiB57.4 GiB55.8 GiB1.26 (p95 1.61)99.6 %6.38 GB/s309 W53.0 tok/s
Flash-Next GGUF IQ3_XXS · strataIdle (loaded)23.7 GiB51.8 GiB57.2 GiB0.09 (p95 0.28)0 %0 GB/s120 W–
Prefill23.7 GiB51.9 GiB57.2 GiB1.34 (p95 2.58)95.3 %12.5 GB/s320 W1,911 tok/s
Decode23.7 GiB51.8 GiB57.2 GiB15.2 (p95 15.6)99.9 %3.81 GB/s352 W92.8 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

What we learned running it

Recipes

Flash-Next · Strata · IQ3_S (best task success)

Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json

Lab decode
78.1 tok/s
Lab prefill
2,178 tok/s
Agentic floor
87.3 tok/s
Load time
43.0 s

Full recipe →

Flash-Next · Strata · Q2_0 (fastest)

Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json

Lab decode
103 tok/s
Lab prefill
2,664 tok/s
Agentic floor
123 tok/s
Load time
25.1 s

Full recipe →

Flash-Next · trellis/SGLang · EXL3 3.05 bpw (Sero's registry route)

Qwen3.8-Flash-Next · launch launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json

Lab decode
56.4 tok/s
Lab prefill
2,240 tok/s
Agentic floor
40.4 tok/s
Load time
267 s

Full recipe →

All Qwen3.8-Flash-Next configs and runs →

Models

Model card

Qwen3.8-27B

huggingface.co/Qwen/Qwen3.8-27B · released 2026-08-05

Fresh-issues slice: Qwen3.8-27B · EXL3 3.00 bpw · 131k launch 36/50; Qwen3.8-27B · registry 208k recipe 39/50 details

Builds we ran

BuildWeightsEngineSize
EXL3 3.00 bpwturboderp/Qwen3.8-27B-exl3
@6fe61ad620ab
SGLang-EXL3 (operator's daily server)–

Resources during execution

CPU cores busy (of 15 cores / 30 threads online)

prefilldecode
00.250.50.7511.2527B EXL3 3.00 bpw · sglang-exl3 · Prefill: 1.10 CPU cores busy1.1027B EXL3 3.00 bpw · sglang-exl3 · Decode: 1.11 CPU cores busy1.1127B EXL3cores busy (mean)

Memory during decode

VRAM (GiB)host RAM: model and buffers (GiB)
051015202527B EXL3 3.00 bpw · sglang-exl3: VRAM 22.6 GiB22.627B EXL3 3.00 bpw · sglang-exl3: host RAM (model and buffers) 5.37 GiB5.3727B EXL3GiB
SetupPhaseVRAMRAM: model & buffersRAM: file cacheCPU busyGPU busyPCIe inPowerSpeed
27B EXL3 3.00 bpw · sglang-exl3Idle (loaded)22.3 GiB5.44 GiB17.1 GiB1.10 (p95 1.38)0 %0 GB/s113 W–
Prefill22.6 GiB5.37 GiB17.1 GiB1.10 (p95 1.27)98.8 %0.18 GB/s384 W984 tok/s
Decode22.6 GiB5.37 GiB17.1 GiB1.11 (p95 1.28)95.3 %2.06 GB/s388 W108 tok/s

Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.

What we learned running it

Recipes

Qwen3.8-27B · registry 208k recipe (SGLang-EXL3, MTP, vision)

Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json

Lab decode
139 tok/s
Lab prefill
836 tok/s
Agentic floor
90.1 tok/s
Load time
113 s

Full recipe →

Qwen3.8-27B · SGLang-EXL3 · 3.00 bpw + MTP

Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json

Lab decode
144 tok/s
Lab prefill
740 tok/s
Load time
131 s

Full recipe →

All Qwen3.8-27B configs and runs →

Models

Model card

GLM-5.3-Flash

huggingface.co/zai-org/GLM-5.3-Flash

Builds we ran

BuildWeightsEngineSize
EXL3 3.05 bpwturboderp/GLM-5.3-Flash-exl3
@332ab457b709
glm53-flash-offload (exact, NVMe tier)125 GB
GGUF IQ2_M / IQ3_XXS / IQ4_XS / Q2_K (bartowski)bartowski/GLM-5.3-Flash-GGUFllama.cpp, ik_llama (1–3 cards)121–178 GB

What we learned running it

Recipes

GLM-5.3-Flash · glm53-flash-offload v4.1 · NVMe tier, 90 GiB (experimental)

GLM-5.3-Flash · launch launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json

Lab decode
11.3 tok/s
Lab prefill
630 tok/s
Load time
113 s

Full recipe →

All GLM-5.3-Flash configs and runs →

Method & Hardware

How every number on this site is produced. Each measurement writes a result record (pass or fail) with the exact command, image digest, weights revision and hardware state.

Speed measurements

  • Lab gates (Sero's registry lab.py): load, chat, reasoning, tool calls, a long-context gate at ~85 % of the configured context, and decode speed over the first 30 s of a streamed answer (C1).
  • Kit sweep (recipe-kit protocol): prefill at 8k–64k with fresh random prompts; decode at 1–4 concurrent streams and at 32k context.
  • TTFT: time to the first token with content at 8k and 32k.
  • Agentic floor: a 10-minute mini-swe-agent coding run; the slowest 60-second window of decode after the first minute.
  • Probe (moetier): GPU, PCIe, NVMe and per-core CPU utilisation every 0.1 s, compared with this box's measured ceilings.

Quality measurements

  • Reference panel: teacher-forced token distributions against an exllamav3 reference (top-1 agreement, KL); engines without logprobs (Strata) cannot be scored this way.
  • Fresh-issues benchmark: private real-world bug fixes; construction below.
  • Public SWE-bench-Lite (30 tasks): kept for reference only; Flash-Next has memorised it.

Fresh-issues benchmark: construction

How the tasks were made

  1. Harvest. A GitHub crawler listed merged pull requests in popular Python repositories and kept only those that close an issue in the same repository, where the issue and the PR were both opened on or after 1 September 2026, the PR targets the default branch, and it changes both source code and tests. Bots, backports, reverts, duplicates, docs-only changes, very large fixes and issues that already contain the fix were removed. At most three tasks per repository.
  2. Clarity review. Each issue was graded by Claude Opus on the SWE-bench Verified rubric: is the issue clear enough to fix, do the hidden tests check only what the issue asks for, and does the issue give the fix away. 328 of 430 attempts failed this review.
  3. Environment. The repository is fetched at the PR's base commit with no later history. An Opus agent, working in a throwaway container without the issue or the fix, writes the install script; the final image is then rebuilt from that script alone and checked: the working tree is clean at the base commit and no later commit exists in it.
  4. Validation. With the hidden tests added, at least one test must fail on the original code and pass with the upstream fix in three separate fresh containers; an empty patch must not pass; flaky tests are dropped; the test run must finish in under 10 minutes.
  5. Freeze. Fifty test tasks and ten development tasks were drawn by difficulty with at most three per repository, the manifest was fixed and its hash published, and the comparison to report was declared before any model ran.

From 18,132 pull requests to 50 tasks

Stage 1, automatic filters (no model involved).

05k10k15k20kPython repos foundGitHub search: ≥ 1,000 stars, > 200 forks, pushed since 1 Sep2,985Repos keptOSI licence, ≥ 60 % Python, active, issue-linked PRs since 1 Sep300Merged PRs crawledmerged PRs created since 1 Sep18,132Issue also after 1 Sepafter the issue_before_cutoff filter: 14,10614,106Code + test changesafter the docs_ci_typing_only filter: 8,7588,758Candidate poolafter the issue_contains_fix_code filter: 5,4505,450Shortlist (≤ 3 per repo)763 from 275 repos763count

From 430 attempts to 50

Stage 2: clarity review, environment build and validation.

0200400600Attempted (ordered list)taken in a frozen, seeded order430Passed the clarity review328 rejected by the clarity review89Validatedgold fix passes 3/3, empty patch fails, hidden tests fail before the fix62Test slice50 test + 10 dev (dev never reported)50tasks
Table view
ReasonRejected
tests check things the issue does not state161
issue gives the fix away95
issue underspecified71
clarity_error1
None12
test patch disables tests7
base:eval error7
no f2p5
setup session timeout3
base:test timeout2
setup disallowed pytest arg:--timeout-method=thread1
gold:eval error1
setup agent gave up1
flaky f2p1

What it tests

Whether a model, driving a coding agent, can read a real bug report, find the cause in an unfamiliar repository and change the code so that the project's own new tests pass, without breaking the tests that passed before.

  • Median issue: 299.5 words; median fix: 7 added lines (90th percentile 26).
  • Median 2 hidden test(s) must flip from failing to passing and 29.5 existing tests must keep passing.
  • 35 of 50 tasks come from repositories that appear in no public SWE dataset.

How a run is scored

  • Agent: mini-swe-agent with native tool calls, one attempt per task, the container has no network.
  • Budget: 60,000 completion tokens per task; sampling temperature 0.7, top-p 0.95, seed 1234, 8,192 tokens per reply.
  • The agent never sees the hidden tests. Its final diff is applied to a fresh copy of the image and the swebench 5.0.2 grader runs pytest -rA on the test files.
  • Solved = every fail-to-pass test passes and every pass-to-pass test still passes.

Integrity

The task list is held back so it cannot leak into future training data. The commitment below is the SHA-256 of the exact manifest bytes followed by a secret 32-byte salt. When the slice is retired, the manifest and salt are published and anyone can check that the scores above came from these tasks.

5371c3b8d1caedc134657a17b86cad94fc7fe13d81365e932d068f5e22a9c710

sha256(manifest bytes || 32 raw salt bytes)

Read these results with

  • Contamination-reduced, not contamination-proof. Dates rule out these exact fixes, not similar ones elsewhere.
  • Opus-curated. Claude Opus chose which issues are clear enough and set up the environments, and Anthropic's service saw the selected tasks: treat the slice as semi-private for Anthropic models.
  • Easier than SWE-bench Verified. The median fix is 7 lines and only 6 tasks are hard, so strong models crowd near the top.
  • One pass, 50 tasks. Differences under about 10 tasks between two configs are within noise.
  • Configurations, not just models. 27B hit 39 output truncations and 67 repetition loops at these sampling settings; part of its gap is how it behaves under this harness.

Hardware, bandwidth measurements, status rules and protocol →

Runs & Data

Every config and run, including failed and crashed ones. Leaderboards are per model; each run page has its command, logs and record.

Qwen3.8-Flash-Next

running

154 records, no qualified eligible config yet

Target: lab decode ≥ 50 tok/s and lab prefill > 1000 tok/s on the same config, passing §5 stability; kit band top-1 ≥ 0.987, KL ≤ 0.0013.

No qualified headline yet. Best so far

Lab decode
103 tok/s ✓
Lab prefill
2,664 tok/s ✓
Stability
none

Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb · current · x8 · GGUF Q2_0

154 runs in 23 configs: ✓ pass 130 ! fail 17 ✕ crash 7 · leaderboard

GLM-5.3-Flash

running

99 records, no qualified eligible config yet

Target: 1× RTX 3090, registry speed gate 15 tok/s (lab C1 decode), §5 stability and quality floor (≥ 80% of the reference solve rate). 3-card configs are a separate recipe.

Best 1 card

Lab decode
11.3 tok/s ✕
Lab prefill
630 tok/s
Quality
not measured
Stability
none

Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 · exllamav3 · ? · current · x8 · below 15 tok/s: documented experiment, no registry PR

Best 3 cards

Lab decode
14.0 tok/s ✕
Lab prefill
189 tok/s
Quality
not measured
Stability
failed ✕

Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · ik_llama.cpp · GGUF IQ2_M · current · 3× x8 · below 15 tok/s: documented experiment, no registry PR

99 runs in 24 configs: ✓ pass 46 ! fail 46 ✕ crash 7 · leaderboard

Qwen3.8-27B

running

24 records, no qualified eligible config yet

Target: 1× RTX 3090, registry speed gate 15 tok/s (lab C1 decode), §5 stability and quality floor (≥ 80% of the reference solve rate). 3-card configs are a separate recipe.

Best 1 card

Lab decode
144 tok/s ✓
Lab prefill
740 tok/s
Quality
not measured
Stability
none

Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 · sglang-exl3 · EXL3 3.00 bpw · current · x8 · registry candidate (if §5 passes)

3 cards: no lab measurements yet.

24 runs in 3 configs: ✓ pass 17 ! fail 7 · leaderboard

Recent runs

StartedOutcomeModelKindConfigKey metricsStageTicket
2026-10-12 11:48 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 99.7 tok/s · lab_prefill_tps 2,664 tok/sT47
2026-10-12 11:48 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gbload_seconds 25.0 sT47
2026-10-12 05:49 AEST✓ pass27Bquality evalsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2solve_rate 0.78 fractionT48
2026-10-12 05:44 AEST✓ pass27Blabsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab_decode_c1_tps 139 tok/s · lab_prefill_tps 836 tok/sT48
2026-10-12 05:33 AEST✓ pass27Bstabilitysglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2decode_window_min_tps 90.1 tok/s · tool_calls 134 callsT48
2026-10-12 05:28 AEST✓ pass27Blabsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab_decode_c1_tps 126 tok/s · lab_prefill_tps 831 tok/sT48
2026-10-12 05:26 AEST✓ pass27Bloadsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2load_seconds 113 sT48
2026-10-12 05:23 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 88.2 tok/s · lab_prefill_tps 2,638 tok/sT47
2026-10-12 05:20 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 101 tok/s · lab_prefill_tps 2,631 tok/sT47
2026-10-12 05:18 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 103 tok/s · lab_prefill_tps 2,644 tok/sT47
2026-10-12 05:17 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gbload_seconds 25.0 sT47
2026-10-12 05:15 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gblab_decode_c1_tps 75.0 tok/s · lab_prefill_tps 2,174 tok/sT47

Funding & Contributions

Contributions upstream

What more hardware would test

Each option targets a limit we measured. Expected gains are hypotheses (mostly simulator estimates) until tested.

TierOptionMeasured constraintHypothesisSuccess criterion
No costMove the model NVMe to a CPU-attached x4 slotGLM NVMe tier saturates the x2 chipset link (3.05 GB/s mean, ≥ 80 % of ceiling 88 % of the time).Simulated 10.8 tok/s instead of 7.0 at a 55 GiB cap; little effect at 90 GiB, where the CPU binds.GLM NVMe recipe ≥ 10 tok/s at 55 GiB.
MediumWorkstation platform: PCIe 4.0 x16 per GPU, 8-channel memory (e.g. EPYC)trellis Flash-Next is PCIe-bound at x8; GLM's CPU expert lane is bound by 2-channel DDR4 and 13 threads.trellis Flash-Next towards Sero's x16 measurement on the same image (kit C1 61 tok/s, lab 67.5); GLM 23–27 tok/s (simulated on his EPYC box).trellis Flash-Next agentic floor ≥ 50; GLM ≥ 20 tok/s.
LargeOne 96 GB Blackwell card (RTX PRO 6000)Experts that do not fit in 24 GB of VRAM stream over PCIe or run on the CPU.GLM 36–40 tok/s simulated (a lower bound; the 3090's fixed per-token time was used). Flash-Next experts mostly resident.GLM ≥ 30 tok/s agentic floor; Flash-Next ≥ 100 tok/s on a 3-bit quant with higher quality.
Not recommendedA 4th RTX 3090None of the tested single-GPU engines pools VRAM across cards.No gain for these engines.—

Costs to be added by the operator. Simulations: moetier on trace G002 (research/moetier-onboarding.md §3).

Runs & Data › Decisions

0001 — Flash-Next llama.cpp comparison quant (T11)

decisions/0001-flashnext-llamacpp-quant.md

0001 — Flash-Next llama.cpp comparison quant (T11)

Decision: use bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f IQ4_XS (97.7 GB) for both the 1-card and the 3-card llama.cpp comparison, on the attested ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312 image. N-gram table (per_layer_token_embd.weight) pinned to CPU with -ot; experts on CPU for the layers that do not fit (--n-cpu-moe N, tuned once per layout), fp16→q8_0 KV only if needed for 128k+.

Why: closest to 4-bit that fits 24 GB VRAM + ~110 GiB RAM on one card (IQ4_XS ≈ 91 GiB); same file serves the 3-card run, so the 1 vs 3 comparison changes only the layout. Q4_K_M (119.6 GB ≈ 111 GiB) would not leave room on 1 card. No MTP (llama.cpp Flash-Next MTP PR #27836 not merged). Not REAP-pruned.

Limits: the kit quality panel speaks the SGLang API only, so this route gets no kit band → it cannot be the Flash-Next headline (SPEC §5); it is a comparison point. Delete after T11 unless it beats trellis.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • IQ4_XS stays the paired 1 vs 3 card file. Q4_K_M only as a single feasibility screen if IQ4_XS is competitive with trellis (disk is the binding constraint).
  • The 1 vs 3 pair freezes occupied context, KV format, batching and workload; --n-cpu-moe / tensor split per layout are reported, and any KV change is a separate comparison.
  • Full digest + shard hashes via tools/models.py; tensor placement of per_layer_token_embd.weight verified from the server log.
  • Eligibility experiment-only (no kit band). Results kept regardless; weights deleted only after manifests exist.
Independent review (0001-flashnext-llamacpp-quant.review.md)
  • AMEND. Keep IQ4_XS as the primary paired comparison, but replace the categorical Q4_K_M rejection with a memory-budget check. The strongest counter-argument is that the decision compares the entire file against host RAM, despite offloading weights to VRAM.
  • Q4_K_M is not demonstrably excluded. 119.6 GB ≈ 111.4 GiB. If 18–20 GiB of weights reside on the GPU and their host pages are reclaimable, host weight residency becomes roughly 91–93 GiB, leaving 17–19 GiB within the stated 110 GiB RAM budget. Loading peaks, pinned allocations, KV and scratch could consume that margin; the supplied numbers do not prove they will.
  • Concrete alternative: use IQ4_XS for T11, with one bounded Q4_K_M feasibility screen at the intended context if the allocation budget supports it. Keep any Q4 results as a separate quant comparison. Do not expand into another tuning grid.
  • Fitting is not evidence of adequate speed. Dual-channel DDR4-3200 has a theoretical 51.2 GB/s ceiling. At 50 tok/s, that permits only 1.024 GB of DRAM traffic per token. If all 6B active LM parameters required fresh ideal 4-bit reads, that is ~3 GB/token and at most 17 tok/s before overhead. Record CPU offload placement and measured decode; this is a conditional bandwidth screen, not a performance prediction.
  • “Only the layout changes” needs qualification. Independently tuning --n-cpu-moe N compares each layout’s selected configuration, rather than isolating card-count scaling. Freeze occupied context, KV format, batching and workload across the pair; report both CPU-expert allocations and tensor splits. Treat any KV-format switch as a separate comparison.
  • Complete provenance before launch. Pin the full image digest already supplied in SPEC §6, not just server-cuda12-b11312. Add the exact GGUF shard list and hashes, immutable launches, and the actual tensor-placement evidence for per_layer_token_embd.weight. The proposed -ot setting remains unverified in this review.
  • The quality limitation is correctly identified, but applies to registry eligibility too. Without the required kit band, this Flash-Next configuration is experiment-only, even if all registry lab gates pass. Neither IQ4_XS nor a speed win establishes quality equivalence.
  • Replace “delete unless it beats trellis.” Define the comparison at matched occupied context and cache state, and retain results regardless of outcome. Delete rejected weights only after hashes and immutable references are recorded; retain any finalist under §9. A useful three-card result need not beat a single-GPU engine to justify publication.

Runs & Data › Decisions

0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)

decisions/0002-glm-quant-plan.md

0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)

Source: bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 (made with llama.cpp b11279, mainline-compatible; MTP layers kept at Q4_0). Unsloth GGUFs excluded: not loadable on mainline until re-uploaded.

LayoutQuantsSizeFit reasoning
1 card (24 GB VRAM + ~115 GiB usable RAM)IQ2_M, Q2_K120.6 / 125.7 GB (112 / 117 GiB)~21 GiB on GPU, rest in RAM
1 card, stretchIQ3_XXS138.8 GB (129 GiB)over RAM by ~10 GiB → mmap paging from NVMe; run once, expect slow, record it
3 cards (72 GB VRAM + RAM)IQ3_XXS, IQ4_XS138.8 / 177.7 GBIQ4_XS ≈ 165 GiB: ~66 GiB VRAM + ~99 GiB RAM; ~4-bit like every accepted GLM recipe

Order (disk is ~750 GB for everything, so sequential): IQ2_M → Q2_K → (IQ3_XXS) → delete the losing 2-bit → IQ4_XS. ik_llama.cpp + MTP runs on the same files.

Quality judged per SPEC §5 (tools gate, 0 parse failures over ≥100 calls, ≥80 % of reference solve rate in the uncapped eval).

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Order: IQ2_M and Q2_K on 1 card → same files on 3 cards → IQ3_XXS on 1 card (tight resident candidate, not presumed paging) and 3 cards → IQ4_XS on 3 cards.
  • Fit is checked by measured placement at the stated context, per GPU; uneven splits watched.
  • Nothing is a finalist before the uncapped quality evaluation; deletion only of rejected candidates, after manifests.
  • MTP on/off is its own variable family; MTP use verified from engine logs and metrics.
  • Publish tool-call counts and reference solve counts next to every quality verdict.
Independent review (0002-glm-quant-plan.review.md)
  • AMEND. Keep the candidate quants, but correct the IQ3_XXS fit claim and make deletion depend on qualification. The strongest counter-argument is that weight capacity does not establish useful speed or quality: the spec already estimates one-card GLM at 8–13 tok/s, below the 15 tok/s registry floor.
  • IQ3_XXS is not automatically a paging experiment. Using the decision’s assumptions: 138.8 GB ≈ 129.3 GiB; subtracting 21 GiB GPU residency leaves 108.3 GiB in RAM, about 6.7 GiB below the stated 115 GiB usable budget. Treat it as a tight resident-memory candidate. Declare paging only after accounting for context, runtime allocations and actual tensor placement.
  • All fit claims need a specified context and measured placement. Approximate host-weight budgets are 91.3 GiB for IQ2_M, 96.1 GiB for Q2_K, and 99.5 GiB for three-card IQ4_XS. Their remaining host margins are roughly 23.7, 18.9 and 15.5 GiB respectively. These exclude additional runtime memory. Likewise, allocating 66 GiB across three cards leaves only 2 GiB per card on average; an uneven split can exhaust one GPU.
  • Budget bandwidth before promising 15 tok/s. Dual-channel DDR4-3200 has a theoretical maximum of 51.2 GB/s. At 15 tok/s, that permits at most 3.41 GB of DRAM traffic per generated token, with a smaller practical allowance. Measure effective bandwidth, expert placement and decode; VRAM capacity alone cannot show that the one-card route clears this constraint. Also, Q2_K is only 4.2% larger than IQ2_M—kernel efficiency and quality could easily determine the winner.
  • “~4-bit like accepted recipes” is precedent, not qualification. The spec’s reported sub-4-bit KLD of 0.28–0.38 makes quality a substantial risk. Establish the reference explicitly: exact EXL3 if loadable, otherwise the highest-bit GGUF that fits. If that is IQ4_XS, retain it through all candidate evaluations. A speed winner cannot be called a finalist before the uncapped quality evaluation.
  • Concrete alternative order: screen IQ2_M and Q2_K on one card, compare those files on three cards, then screen IQ3_XXS on both layouts and IQ4_XS on three cards. Run uncapped quality evaluation for promising configurations, then qualification for survivors. Give one-card IQ3_XXS an ordinary residency check before labelling it “stretch”; make three-card IQ3_XXS an explicit comparison point.
  • Replace “delete the losing 2-bit” with “delete a rejected candidate.” All four listed quants total 562.8 GB, leaving about 185 GB of the specified 748 GB free before Qwen weights, images and other artifacts. Build the combined inventory first. Preserve hashes and file references before deletion, and retain qualifying finalists under §9; being slower than another quant does not itself establish rejection.
  • Shared files do not establish shared MTP support. Verify that each pinned engine loads this exact revision and actually uses the retained MTP layer. Compare MTP off/on as a separate variable family; record accepted tokens/s, acceptance rate, settings and quality. Keep locally built ik_llama.cpp results experiment-only until its image provenance becomes eligible.
  • The decision’s quality summary omits essential acceptance conditions. Require the rolling 60-second decode floor, three qualification runs, cold-cache reset, no OOM/crash/repetition, and all six lab gates. Zero failures in 100 tool calls satisfies the spec but still gives an approximately 3% one-sided 95% upper bound on failure probability. The solve-rate test is also coarse: with four reference solves, all four are required; with five, four suffice. Publish these counts alongside occupied context and exact launch provenance.

Runs & Data › Decisions

0003 — Harness interpretations of SPEC §5 (repetition, parse failures)

decisions/0003-harness-interpretations.md

0003 — Harness interpretations of SPEC §5 (repetition, parse failures)

Context: the T06a harness was exercised for 10 min against the operator's daily Qwen3.8-27B (EXL3 3 bpw, SGLang): 73 tool calls, 0 malformed tool calls, but 12 repetition hits — genuine reasoning loops (the same multi-paragraph block 3× inside one reasoning field) on sympy-21612 at 60–83k occupied tokens. Under §5 as written, the operator's everyday model fails stability.

Interpretations implemented by the harness:

  1. Repetition, per field: the 64-token ×3 rule is evaluated within each generated field (reasoning, content, each tool call's arguments), not over the concatenation, because code drafted in reasoning and then emitted in a tool call trips the joined check without being a loop.
  2. Parse failures, strict: besides invalid tool-call JSON and unparsed tool-call markup, every agent format error counts — including a response truncated at max_tokens (8192) with no tool call. Each response counts once; causes are split in detail.
  3. Same-call streak (≥4 identical consecutive tool calls) resets at each task attempt.

Proposed (pending data, operator decides): keep §5 strict for this first pass and record all hits with causes; after T10/T20 produce real data, revisit whether "no repetition loops" should mean "no unrecovered loop" (a loop that ends the response by hitting max_tokens or ends the task) — not changed now.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • SPEC §5 amended: per-field repetition (cross-field diagnostic only), detector frozen for T10/T20, streak resets only on a fresh conversation (each task attempt starts a new mini-swe-agent conversation, so the harness already complies).
  • Three error categories: parse_failures (malformed tool calls only — the qualification criterion), output_truncations, agent_format_errors, all published.
  • No relaxation of "no repetition loops" now. Recovery and wasted-token figures to be published as supplementary data; any protocol revision later is prospective and does not re-grade earlier results.
  • Denominators always shown next to zero-failure claims.
Independent review (0003-harness-interpretations.review.md)
  • AMEND. Keep the strict no-loop gate, but distinguish detector clarification from changes to eligibility. The strongest objection is that the harness simultaneously narrows repetition detection and broadens parse failures, while claiming to preserve §5.
  • Per-field repetition is defensible, but requires an explicit spec amendment. “Within one response” currently includes repetitions across fields. Adopt: “Within reasoning, content, or each tool-call argument field, independently, using the served model’s tokenizer.” Retain cross-field matches as diagnostics so the excluded cases remain auditable.
  • Do not label every agent-format error a tool-call parse failure. Invalid JSON and malformed tool markup qualify; an otherwise valid response exhausted at max_tokens without a tool call is a truncation or agent-protocol failure. Record tool_parse_failure, agent_format_error, and output_truncation separately. If all must disqualify, add that requirement explicitly to §5.
  • Resetting the same-call streak is appropriate only at an independent task attempt. Specify that a fresh agent conversation resets it; a retry continuing the same conversation does not. Otherwise attempt boundaries can conceal four consecutive identical calls.
  • The daily-model result does not justify relaxing the gate. Twelve genuine loops establish that this particular screening run fails the agreed criterion. They do not establish that the criterion is unsuitable for the two target models. The observed 60–83k occupancy also falls below the ≥128k requirement for “agentic-usable.”
  • Reject “unrecovered loops only” as the qualification alternative. Recovery does not erase wasted generation or latency. At exactly the specified decode floors, an 8,192-token response takes approximately 164 seconds for Flash-Next or 546 seconds for GLM, excluding prefill—27% or 91% of a ten-minute run. Publish recovery and wasted-token measurements as supplementary metrics while retaining zero detected loops for qualification.
  • The parse evidence is preliminary. Seventy-three calls falls short of the ≥100-call minimum. Under an independent binomial assumption, 0/73 failures still gives a one-sided 95% upper failure-rate bound of approximately 4.0%; 0/100 gives 3.0%. Report the denominator without presenting zero observed failures as strong reliability proof.
  • Concrete alternative: ratify the field boundary and separate error categories before target comparisons; freeze that detector version for T10/T20; preserve each offending response and count affected responses separately from detector matches. Use later data to propose a prospective protocol revision, with existing results retaining their original qualification status.

Runs & Data › Decisions

0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)

decisions/0004-core2-quarantine.md

0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)

Evidence: two hard resets on 2026-10-03 (04:57 near-idle, 06:56 during T10 model load). Each next boot logged an uncorrectable machine check in bank MC5 (Execution Unit) — CPU:18 MC5_STATUS[-|UE|MiscV|-|PCC|TCC|SyndV] and CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV]. CPUs 2 and 18 are SMT siblings of physical core 2. No OOM, no GPU Xid, no kernel panic text: the core faulted and the platform reset. Likely cause: per-core Curve Optimizer/PBO undervolt too aggressive for that core, or core degradation (RMA case if BIOS is stock; BIOS 5101).

Decision (operator, option 2): take CPUs 2 and 18 offline for the whole campaign (resets at reboot; hygiene.sh check refuses to benchmark while they are online). Every record carries host.cpus_online/cpus_offline. llama.cpp and ik_llama launches use 15 threads. Then validate with tools/stability/cpu_check.sh (all-core AVX2, per-core single-thread boost bursts, all-core openssl) and look for new machine checks.

Consequence for results: numbers are for a 15-core 5950X. CPU-bound routes (GLM experts on CPU) are affected ~6 %; GPU/PCIe-bound trellis decode should not be. Any result produced before this decision does not exist (T10 never completed).

Check result (2026-10-03 07:25–07:40)

cpu_check.sh 300 20 180 on the 30 online CPUs: 0 machine checks, no reset (log: logs/cpu_check-core2-offline.log). Not conclusive for the near-idle failure mode; the hygiene check keeps the quarantine enforced for every run.

Revision (2026-10-03 10:30) — quarantine lifted with a tripwire

  • Operator BIOS check: PBO was Auto (ASUS stock); Curve Optimizer set to all cores, positive 0 (= no offset). So the CPU was effectively at stock when it faulted; a weak core 2 remains the leading hypothesis.
  • TARGET_CPUS="2 18" cpu_check.sh 300 20 180 with all 32 CPUs online: all-core AVX2, per-core bursts, 5 min sustained on CPU 2 and 18 each, 5 min on/off bursts on each, all-core openssl — 0 machine checks, no reset.
  • Not conclusive (both crashes came after hours of uptime). Core 2 is back online for full capacity; hygiene.sh check refuses to benchmark and re-offlines CPUs 2,18 if any machine check appears in the current boot; every record carries host.machine_checks_this_boot. llama.cpp/ik_llama launches back to 16 threads.
  • T10 ran on 15 cores (recorded in its host block). Trellis decode is PCIe-bound; one 16-core re-check is planned.
  • If it recurs at stock: keep core 2 offline for the campaign; RMA candidate.

Revision (2026-10-04 08:05) — permanent quarantine

  • Third fatal reset with core 2 online at stock: 2026-10-04 06:14, ~20.5 h into the boot, during the 16-core T10 re-check model load (CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], logged at the 06:54 boot). Same pattern as the 2026-10-03 06:56 crash (Flash-Next load) after a long uptime.
  • Core 2 stays offline for the rest of the campaign, enforced at boot by local-ai-bench-quarantine.service (tools/box/install.sh) and by hygiene.sh check (CPU_QUARANTINE defaults to "2 18"). All llama.cpp/ik_llama launches use 15 threads. The 16-core T10 re-check is dropped; all results are for a 15-core 5950X.
  • Stock settings + repeated MC5 UE on one core: AMD RMA candidate (operator's call).

Runs & Data › Decisions

0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)

decisions/0005-ik-llama-glm-tool-parsing.md

0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)

Finding (T22b, IQ2_M, 1 card): the model emits well-formed GLM tool calls (<tool_call>bash<arg_key>command</arg_key><arg_value>…</arg_value></tool_call>) but ik_llama's server returns them as plain content with empty tool_calls: lab tools gate fails; stability run 0 tool calls / 47 parse failures (unparsed_markup). ik_llama builds tool parsers from the chat template (common/chat-auto-parser*); mainline llama.cpp b11312 parses the same GGUF correctly (tools gate passed in T20b). No matching ik_llama issue/PR found (search 2026-10-04).

Speed is unaffected and recorded: decode 10.0 tok/s (mainline 9.6), prefill 151 lab / 203 TTFT-8k (mainline ~80).

Decision: keep the ik_llama route as speed evidence only (experiment-only, fails §5 tools); do not patch ik_llama in this campaign — single-card GLM is ~10 tok/s on every route, below the 15 tok/s gate, so tool parsing would not change the outcome. Phase 3 option: upstream bug report with the captured response (artifacts/runs/…ik_llama…stability/proxy/parse_failures/).

3-card follow-up (2026-10-04, GPU2-first order, decision 0006)

  • No MTP, --fit-margin 4096: lab decode 13.4 / 14.0, agentic 13.3 tok/s steady (best GLM decode on this box), prefill 185–283; tool calls still unparsed (68 agent-format errors), tools gate fails.
  • MTP, --fit-margin 7168 (4096 OOMed creating the MTP context): lab decode 10.2 (acceptance 51 %), agentic 15.6 whole-run before the server crashed mid-run. The extra margin for the draft context pushes experts to host RAM.
  • Route parked: experiment-only speed evidence. The most interesting number for 0xSero is 13–14 tok/s without MTP; a tool-parser fix in ik_llama plus an attested image would be needed before it could be a 3-card candidate.

Runs & Data › Decisions

0006 — PCIe lane layout: what can be reconfigured, and what it buys

decisions/0006-pcie-layout.md

0006 — PCIe lane layout: what can be reconfigured, and what it buys

Operator question (2026-10-04): can the PCIe lanes be reconfigured to get better model performance?

Measured topology (lspci, nvidia-smi, T05a)

DeviceAttachLinkMeasured H2D
GPU0 3090 04:00.0X570 chipset, behind a Gen4 x4 uplink shared with the KC3000 1 TB NVMe (root + all model files), NIC, BMC, SATA, USBGen4 x8 to the chipset6.0 GB/s
GPU1 3090 0b:00.0CPU lanes, slot 2 (x8/x8 split) — drives the displayGen4 x8not measured (display)
GPU2 3090 0c:00.0CPU lanes, slot 1 (x8/x8 split)Gen4 x813.5 GB/s
990 EVO Plus 4 TBCPU lanes (M.2_1, Gen4 x4)unused: holds another OS install

Already optimal and not a lever: every link trains at Gen4 (16 GT/s); Resizable BAR is on (32 GB BAR1); ASPM is disabled on every GPU link. AM4 has 24 usable CPU lanes (16 slots + 4 M.2 + 4 chipset uplink); bifurcation or risers cannot add lanes, so the only real choices are which card sits where and how many cards share the 16 slot lanes.

Where PCIe bandwidth limits each route (evidence from our runs)

  1. Flash-Next trellis (experts streamed from host RAM to one GPU): decode is PCIe-bound. 0xSero's data: ~26 GB/s → 62.7 tok/s, 14–16 GB/s → 53.2. Ours at 13.5 GB/s: lab 49–55, kit C1 46.9, agentic 43. x16 should add ~15–25 %.
  2. llama.cpp/ik_llama prefill with experts in RAM is PCIe-bound (CPU-resident weights are copied to a GPU for large batches). Evidence: on GPU2, raising -b/-ub 512 → 2048 lifted GLM IQ2_M prefill 85 → 200 tok/s (each copy serves 4× more tokens). And 3 cards prefilled slower than 1 card (158 vs 200), consistent with the copy going to CUDA0 = GPU0, the 6 GB/s chipset card.
  3. llama.cpp decode with experts on the CPU is DDR4-bound, not PCIe-bound (GLM ~9.5 tok/s on 1 card, 12.3 on 3 cards, from less RAM-resident weight). PCIe changes nothing here.

Options

#ChangeCostExpected effect
S13-card llama.cpp with the CPU-attached GPU2 as CUDA0 (CUDA_VISIBLE_DEVICES=2,1,0 in the container)none3-card prefill back to ≥ 1-card level (~200+), no decode change
S2Larger ubatch (-b/-ub 4096) on llama.cpp routesnone (more VRAM for compute buffers)prefill up to ~1.5–2× again; watch VRAM (the IQ2_M 3-card run already OOMed on CUDA0)
P1One card alone in slot 1 at x16 (T12, already planned)operator moves cardsFlash-Next decode +15–25 % (the only route to ≥ 50 headline at full context); llama.cpp prefill ~2× vs x8; GLM exact mode at the bandwidth its 12.6 tok/s reference used
P2Plug the monitor into GPU0 (chipset card) instead of GPU1move one cableGPU1 + GPU2 (both CPU x8) free for 2-card runs without going headless; desktop stays usable during single/2-card work
P3Model files on the CPU-attached 990 EVO Plus instead of the chipset KC3000needs the operator's OK (other OS lives there)removes model loads and Flash-Next n-gram NVMe reads from the chipset uplink that GPU0 also uses; small effect except for 3-card runs with GPU0
P42 cards on CPU lanes (x8/x8), third card removed or idlenone (GPU1+GPU2)avoids the 6 GB/s card; less VRAM (48 GB) so more experts in RAM

Decision (proposed)

  • Run S1 and S2 now in the current layout (software only, attributable, one variable family each), on GLM IQ2_M: 3-card GPU2-first at ub2048 (vs the existing GPU0-first record), then 1-card GPU2 at ub4096 (vs ub2048).
  • Keep P1 (T12) as the main hardware lever; in the x16 session also rerun the best GLM llama.cpp 1-card config so the x8→x16 effect on prefill is measured directly.
  • Recommend P2 to the operator at the T12 visit (same visit, one cable). P3 only if the operator wants to free the 990 EVO Plus; not worth touching another OS for this campaign.
  • Publish this as a site methodology section with the before/after records.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted, plus operator rule

  • Bandwidth effects are hypotheses until measured. Expected-gain numbers above are estimates, not claims; x16 is the principal hardware hypothesis for Flash-Next, not a guarantee of qualification.
  • S1 is a fresh paired comparison, both headless, identical except card order: IQ2_M 3-card ub2048 with a 2 GiB --fit-target margin (the default-margin run OOMed on CUDA0 after 62 min), GPU0-first vs GPU2-first (CUDA_VISIBLE_DEVICES=2,1,0; the container sees the cards in PCI order). Every TTFT step now logs per-GPU PCIe rx/tx (nvidia-smi dmon -s t) to show which card receives the weight copies. Decode changes are reported, not assumed.
  • S2: 1-card GPU2 ub2048 vs ub4096, both at 15 threads (the old ub2048 record ran 16 threads), peak VRAM and OOMs recorded.
  • P1: rerun the best 1-card GLM llama.cpp config at x16 with the same bandwidth procedure (x8 control exists).
  • Operator rule (2026-10-04): GPU0 is used last. Single-card runs use GPU2; 2-card runs GPU2+GPU1; 3-card runs order the cards GPU2, GPU1, GPU0. Pending S1 evidence, all new multi-card launches use the GPU2-first order.
  • P2 offered to the operator at the T12 visit; P3 deferred (would need a §9 exception and touches another OS).

S1 result (2026-10-04 12:15–13:05 AEST) — card order, GLM IQ2_M 3-card ub2048, 2 GiB fit margin, headless

GPU0 first (control)GPU2 first (CUDA_VISIBLE_DEVICES=2,1,0)change
TTFT prefill 8k / 32k159 / 169 tok/s240 / 242 tok/s+51 % / +43 %
lab prefill153 tok/s218 tok/s+43 %
lab decode C112.2 tok/s11.8 tok/s−3 %
PCIe rx during TTFT (avg / peak)GPU0 3.9 / 6.5 GB/s (link-saturated), others ~0.3GPU2 5.7 / 14.3 GB/s, others ~0.5

The host→device weight copies for prefill go to CUDA0; putting a CPU-lane card there lifts prefill ~1.5×. Decode is within noise of unchanged (slightly lower). The GPU2-first config also finished a clean 10-min agentic run (11.3 tok/s, 24 tool calls, 0 parse failures, 0 loops) and passed the post-run gates without the OOM that hit the default margin. Rule confirmed: GPU0 last.

S2 result (2026-10-04 13:05–14:23 AEST) — ubatch, GLM IQ2_M 1 card GPU2, 15 threads

ub2048 (control)ub4096change
lab prefill201 tok/s288 tok/s+43 %
TTFT prefill 8k / 32k213 / 222321 / 316+51 % / +42 %
lab decode C19.59.6unchanged
VRAM at ready21.9 GB22.9 GB+1 GB, no OOM in a 10-min agentic run

The 15-thread control matches the earlier 16-thread ub2048 record (9.5 / 200), so the core-2 quarantine costs nothing measurable on this route. Gain > 3 %, so the search continues: ub8192 on 1 card, and ub4096 on the GPU2-first 3-card config.

S2 step 2 (2026-10-04 16:13–16:30 AEST) — ub8192, 1 card GPU2

lab prefill 370 (+28 % vs ub4096), TTFT 384 / 404, but lab decode 9.1 (−5 %): the larger compute buffer makes --fit keep more experts in host RAM (VRAM at ready 22.2 GB). Prefill and decode now trade off, so the batching search stops here: ub4096 is the balanced setting (decode unchanged, prefill +43–51 %); ub8192 is the prefill-maximising variant, published as such.

  • 3-card GPU2-first, ub2048 → ub4096 (16:30–16:58): lab prefill 218 → 258 (+18 %), TTFT 240 → 298 (+24 %), lab decode 11.8 → 11.2 (−5 %). Same prefill/decode trade-off as 1 card at ub8192; both published, no further steps.
Independent review (0006-pcie-layout.review.md)
  • AMEND. Proceed with S1, a bounded S2 test, and T12, but describe the bottlenecks and speedups as hypotheses. The strongest counter-argument is that the current runs change batching, weight placement, and GPU count; they do not isolate PCIe bandwidth.
  • Flash-Next’s +15–25% estimate is plausible; “PCIe-bound” overstates the evidence. The reference gain is only 17.9% despite bandwidth rising from 14–16 to 26 GB/s. A simple fixed-work Amdahl model attributes roughly 33–39% of baseline time to transfers, predicting 20–25% from doubling bandwidth. That model remains unverified across machines and contexts.
  • x16 does not establish qualification. Applying +15–25% to the measured agentic 43 tok/s gives 49.5–53.8 tok/s, and averages cannot prove every required rolling window clears 50. Replace “the only route to ≥50 headline” with “the principal hardware hypothesis”; qualification still requires the same config to pass occupied-context, prefill, quality, and stability requirements.
  • The prefill explanation is not established. Increasing batch size fourfold produced 2.35× throughput, from 85 to 200 tok/s. Transfer amortization, better GPU utilization, and execution overhead can all explain this. Likewise, 158 versus 200 tok/s across three versus one GPU does not identify CUDA0 as the cause. Replace “PCIe-bound” and “~2× at x16” with measured observations and an explicit hypothesis.
  • Make S1 a fresh paired comparison. Run GPU0-first and GPU2-first at ub2048 with identical weights, offload settings, tensor split, context, and cache state; both must be headless. Select devices by UUID, record their logical CUDA mapping, and verify which GPU actually receives the expert transfers. Reordering devices alone does not demonstrate the claimed routing change. Report decode changes rather than assuming none.
  • Bound S2 by memory and measurement. Compare one-card GPU2 ub2048 with ub4096 while holding the other settings fixed; record peak VRAM and OOMs. Remove the promised “1.5–2× again.” If both batch and ubatch change, identify the comparison as a batching-family change without attributing its effect specifically to ubatch.
  • Keep P1, with a matching x8 control. Run the exact same best one-card GLM configuration before and after the move, alongside bandwidth measurements under a defined, identical procedure. The 12.6 tok/s exact-mode reference already used x16 and remains 16% below the 15 tok/s floor; matching its link does not imply registry eligibility.
  • P2 is useful, subject to the display rule. Moving the display to GPU0 enables display-free GPU1/GPU2 comparisons. Three-card tests still require logout and display: off; the x16 benchmark must also use a GPU that is not driving the display.
  • Defer P3 and public publication as proposed, but clarify scope. Storage contention has not been measured, and changing active-model storage would require an explicit exception to §9. Put the methodology and every before/after result in the private preview now; public publication remains Phase 3 after operator and log review.

Runs & Data › Decisions

0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs

decisions/0007-glm-current-layout-finalists.md

0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs

Facts (all current-layout GLM records, 2026-10-03/04)

RouteCardsLab decodeLab prefill10-min run (tool calls / parse failures / loops / solved)
llama.cpp IQ2_M ub409619.628827 / 0 / 1 / 1 of 2
llama.cpp IQ2_M ub2048, GPU2 first311.821824 / 0 / 0 / 1 of 2
llama.cpp IQ2_M ub2048, GPU0 first312.315826 / 0 / 0 / 1 of 2 (server later OOMed)
llama.cpp IQ3_XXS ub2048310.413222 / 0 / 1 / 1 of 2
llama.cpp IQ4_XS ub2048, GPU2 first38.315625 / 0 / 0 / 1 of 2
llama.cpp Q2_K / IQ3_XXS19.2 / 8.378 / 650–1 solved
ik_llama IQ2_M (± MTP)18.2–10.0 (MTP agentic ~12.6)~1500 tool calls: GLM tool calls not parsed (0005)

Every route is below the 15 tok/s registry gate on 1 card and on 3 cards (best 12.3). Decode is DDR4-bound: more cards only help by holding more experts in VRAM. Pending in the queue: IQ2_M 1-card ub8192 and 3-card ub4096 (prefill-only), ik_llama 3-card ± MTP (speed evidence only).

Proposal

  1. No current-layout GLM finalist. §5 qualification requires decode ≥ 15 tok/s in every window, so the 3 × 10-min qualification runs would fail by construction. T25a records "no finalist: below the floor" for every current-layout config instead of running them. All configs stay published as documented experiments (§6 Phase 2).
  2. One uncapped quality evaluation, of the best mainline weights, in the x16 session. Solve rate depends on weights, engine and sampling, not on the PCIe layout; the layout only changes how long the evaluation takes. So the T07 candidate evaluation of llama.cpp IQ2_M runs on the single x16 card next to the T23 reference (exact mode, or the highest-bit GGUF that fits if exact mode cannot load), with the identical task list, sampling and 60k-token budget. Each evaluation is estimated at 4–7 h (measured ~3–4k completion tokens per 10 min).
  3. No uncapped evaluation for IQ3_XXS / IQ4_XS / Q2_K / ik_llama. They are slower than IQ2_M and cannot become candidates; their 10-min screening solve counts are published labelled "screening only". IQ4_XS only fits on 3 cards, so measuring it would cost ~6 h of current-layout time for an informational number.
  4. Consequence: T12 (card move) can happen as soon as the current queue finishes (~3.5 h), not after ~1 day of current-layout quality runs. After the queue, IQ3_XXS and IQ4_XS are marked rejected and deleted (manifests kept) to make room for the x16 weights (Flash-Next 2.05/4.05 bpw: 63/108 GB; GLM exact EXL3: 125 GB).

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Below-floor is measured, not inferred from lab C1: every current-layout llama.cpp config with a 10-min run has a measured rolling 60-s generation-only minimum below 15 (IQ2_M 3-card 11.0–11.4, IQ2_M 1-card 8.8–9.1, IQ3_XXS 3-card 9.9, IQ4_XS 3-card 7.8). Configs without a stability run (prefill-only S1/S2 controls, ub8192) are recorded as "qualification skipped: subfloor lab C1; rolling floor not measured". T25a closes with no current-layout finalist once the pending queue (ik_llama 3-card ± MTP, ub follow-ups) is in; ik_llama stays experiment-only (0005).
  • Quality evaluation is conditional. At x16: measure IQ2_M 1-card first; only a config that qualifies (§5 incl. the rolling floor, ≥ 100 cumulative tool calls, post-run gates) gets the uncapped T07 candidate evaluation. An optional quality study of a non-qualifying config is labelled informational. Quality results apply to the tested configuration; layout independence is not assumed.
  • Runtime estimate corrected: the 60k budget is per task; worst case 2.5–3.3 h per task, 12–33 h per evaluation (less when tasks submit early). Budget accordingly; this is another reason to evaluate qualifiers only.
  • Labels: IQ4_XS, IQ3_XXS (3-card), Q2_K: "rejected for speed, quality unmeasured". IQ2_M 3-card GPU0-first: OOM after 62 min (default fit margin). IQ2_M/IQ3_XXS repetition hits are published with the run.
  • Storage: keep IQ2_M (x16 rerun) and IQ3_XXS (highest-bit GGUF that fits 1 card = fallback reference if exact mode cannot load); delete IQ4_XS after the queue, manifests and hashes kept for reacquisition.
Independent review (0007-glm-current-layout-finalists.review.md)
  • AMEND. Finish the queue and move to x16 without spending a day on current-layout quality runs. The strongest objection is that the proposal treats lab C1 as proof of failing a different metric, then schedules a “candidate” quality evaluation for a configuration that has not qualified.
  • “Fail by construction” is unsupported. Lab decode measures the first 30 seconds; §5 requires generation-only decode over rolling 60-second windows, ignoring the first minute. Best C1, 12.3 tok/s, is 18% below the floor and needs 22% uplift to reach 15. That warrants deprioritization, but not a claimed measurement of the floor. Use existing screening traces to establish a failing window; otherwise record “qualification skipped: subfloor lab C1; rolling-floor eligibility unverified.”
  • Make x16 quality evaluation conditional. §5 says “Finalists only.” First measure IQ2_M on x16; if promising, complete qualification, including ≥100 cumulative tool calls and all six post-run gates. Run the uncapped comparison only if it qualifies. If it remains below 15, finish it as a documented experiment under Phase 2. Any optional quality study should be explicitly labelled informational.
  • Layout independence is a hypothesis, not a guarantee. Fixed weights and token budgets remove much of the speed bias, but device placement can change numerical execution and sampled trajectories; timeouts, OOMs and truncation can also affect outcomes. The x16 evaluation directly supports the tested x16 configuration. Transfer its quality result to another layout only with matching engine, settings and evidence of equivalent behavior.
  • IQ2_M is the speed leader, not established “best weights.” The 10-minute results cannot rank quantization quality. IQ4_XS could be slower yet substantially more accurate—a relevant possibility given §12’s sub-4-bit quality risk. Skipping its uncapped evaluation is reasonable because it has no demonstrated eligibility; label it rejected for speed, with quality unmeasured.
  • The runtime estimate needs its budget scope stated. At 3–4k completion tokens per 10 minutes, exhausting 60k tokens per task takes 2.5–3.3 hours per task. Across 5–10 tasks, that is 12.5–33 hours per evaluation, or 25–67 hours for candidate plus reference, assuming similar throughput. Early completion can reduce this. If 60k means an aggregate budget, specify the per-task allocation; “4–7 hours” currently lacks a clear basis.
  • Keep T25a provisional until the queue finishes. Pending ik_llama 3-card/MTP measurements could change the speed ranking, although parsing and image eligibility remain blockers. Existing IQ2_M repetition and GPU0-first OOM evidence also need explicit failure labels. Faster decode alone does not make these configurations finalists.
  • Treat “DDR4-bound” as a working diagnosis. The table does not isolate memory bandwidth from PCIe, placement, batching or engine overhead. The one-card 9.6 tok/s result needs 56% uplift to reach 15, so an x16 rescue is uncertain. Measure it before committing to expensive quality work.
  • Concrete replacement: finish pending measurements → close current-layout records with measured failures or justified skips → move cards → measure and qualify x16 routes → evaluate qualifying candidates against the specified reference. Delete rejected weights after preserving hashes and immutable references, while reserving or budgeting reacquisition of the highest-bit GGUF fallback. Keep results private until Phase 3.

Runs & Data › Decisions

0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate

decisions/0008-flashnext-3card-llamacpp.md

0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate

Evidence (T11, 2026-10-04 15:38 AEST, IQ4_XS, GPU2,GPU1,GPU0 order, 2 GiB fit margin, -b 2048 / -ub 512 defaults)

  • Lab: all six gates pass, decode C1 57.1 / 56.9 tok/s (before / after the agentic run), lab prefill 608 / 605.
  • TTFT prefill: 987 tok/s at 8k, 852 at 32k.
  • 10-min agentic run: 70 tool calls, 0 parse failures, 0 loops, 1 of 2 solved, whole-run decode 45.4, lowest 60-s window 38.6 (fails the 50 floor; decode falls as the context grows).
  • VRAM at ready 65.5 GB of 72; weights 97.7 GB, so ~30+ GB (incl. the n-gram embedding table) stays in host RAM.
  • For comparison, the 1-card trellis/SGLang baseline at x8: lab 49–55, kit C1 46.9, agentic 43.

Per SPEC §6 a 3-card config is a separate, clearly labelled recipe; llama.cpp routes are judged by the registry lab and §5 (the kit's SGLang-only panel cannot score a GGUF, so it is "kit band not measurable").

Proposal (current layout, before T12; one variable family per step, stop rule 3 × < 3 %)

  1. Re-download IQ4_XS (deleted at 16:13 per plan; manifest kept, 97.7 GB, ~15 min).
  2. Prefill: -b/-ub 2048, then 4096 (GLM showed +135 % then +43 %; target lab prefill > 1000).
  3. Decode floor: the floor fails late in the run as context grows. One quant step down to IQ3_M (93.2 GB) or IQ3_XXS (88.0 GB) puts more experts in VRAM; run the better prefill setting from step 2 on it. Quality cost is unmeasured by the kit for GGUF; report solve counts and parse failures, label "quality: screening only".
  4. If a config reaches lab decode ≥ 50, lab prefill > 1000 and holds the 50 floor in a 10-min run, it becomes a 3-card finalist: §5 qualification (3 runs incl. cold and max context, ≥ 100 tool calls) before T12.
  5. Disk: the x16 downloads (296 GB) wait until this route is done and its rejected GGUFs are deleted.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Route is experiment-only unless an accepted scoring path establishes the Flash-Next kit band for a GGUF; kit band recorded as unavailable, registry/headline eligibility unestablished. The T11 config is a failed screening config (floor −29.5 %, lab prefill −65 % vs targets), reported per constraint.
  • Batching steps, predeclared, judged by official lab prefill (decode and VRAM regressions checked): A -b 2048 -ub 2048, B -b 4096 -ub 4096, both vs the existing -b 2048 -ub 512 record. (GLM evidence says the ubatch sets the host→GPU copy amortisation, so b and ub move together.)
  • Before any quant change: decode vs occupied context from the existing T11 agentic trace (per-request generation speed against prompt tokens). IQ3_M is screened only if that shows a residency/transfer limit, not attention cost; its step is judged by the minimum rolling-window decode at matched occupancy. Expert/n-gram placement and memory use recorded from the server log.
  • Qualification only for a speed-passing config, naming its maximum occupied context (≥ 4k headroom), cold resets, post-run gates, ≥ 100 tool calls cumulative. Phase timebox kept; x16 is not deferred beyond this route's steps.

Decode vs occupied context (T11 3-card agentic trace, 55 requests, generation-only tok/s per request)

prompt tokensper-request decode
1.5k–6k (django task)35–51, typical ~42
6k–20k30–49, typical ~40
20k–31k30–41, typical ~38
31k–39k28–38, typical ~33

Two separate gaps to the 50 floor: a base gap (~42 tok/s already at 2–6k tokens, vs lab C1 57 measured over one long generation; agentic requests are short, 70–600 tokens) and a context slope of roughly −20 % from 5k to 39k. One quant step (IQ3_M, −4.6 % of bytes; IQ3_XXS −9.9 %) cannot close a 25–30 % gap even if every saved byte moved experts to VRAM. IQ3_M step dropped; the predeclared batching steps still run (prefill evidence, cheap). After them the route closes as experiment-only and its GGUF is deleted for the x16 weights.

Batching result (2026-10-04 18:00–18:23 AEST) — route closed

-b / -ublab decode C1lab prefillTTFT 8k / 32k
2048 / 512 (T11 record)57.1608987 / 852
2048 / 204844.3 (−22 %)717 (+18 %)1067 / 931
4096 / 409635.0 (−39 %)713980 / 881

Larger ubatches buy little prefill and cost a lot of decode: the bigger compute buffers make --fit move experts to host RAM. No setting reaches lab prefill > 1000 while keeping decode ≥ 50, and the agentic floor already failed at the default. Route closed: experiment-only, best config = the default-batch T11 record (all six gates, lab 57.1, agentic floor 38.6). GGUF deleted for the x16 weights (manifest kept).

Independent review (0008-flashnext-3card-llamacpp.review.md)
  • AMEND — retain this as a separate 3-card experiment, but remove the promised path to registry eligibility until quality is measurable. The strongest counter-argument is contractual: §5 explicitly requires the Flash-Next kit band for both the headline and registry PR. “Kit band not measurable” cannot establish a pass. §12’s lab-only tooling note creates an ambiguity; it does not explicitly waive that requirement.
  • T11 is a failed screening config despite passing all six lab gates. Its worst window needs 29.5% improvement to reach 50 tok/s; lab prefill needs approximately 65% improvement to exceed 1000. Lab C1 at 56.9–57.1 does not compensate for either failure. Report these as separate constraints.
  • The prefill experiment is reasonable; the GLM gains are insufficient evidence for its expected payoff. Specify exact settings rather than “-b/-ub 2048, then 4096.” A concrete sequence is baseline -b 2048 -ub 512, then -b 4096 -ub 512, then -b 4096 -ub 1024, measuring identical occupied contexts and fresh prompts. Select using official lab prefill, and check for decode or memory regressions.
  • The quantization hypothesis needs a residency measurement. IQ3_M saves 4.5 GB (4.6%); IQ3_XXS saves 9.7 GB (9.9%). Against roughly 30 GB remaining on the host, those represent at most approximately 15% / 32% reductions, assuming every saved byte reduces host residency. They do not prove that more active experts become GPU-resident or that transfers shrink proportionally. Record expert placement, n-gram placement, and actual memory use.
  • “Decode falls as context grows” establishes correlation, not the bottleneck. Before changing quant, report generation speed against occupied context, cache state, and GPU placement. Repeat controlled requests at representative early and late occupancies using IQ4_XS. Otherwise a smaller quant could obscure a context-dependent attention or transfer limitation.
  • Solve counts cannot replace the required quality comparison. One of two solved tasks is weak screening evidence, especially under a wall-clock cap where faster configurations receive more generation opportunities. Keep “quality: screening only,” but explicitly mark the kit band unavailable and registry/headline eligibility unestablished for every GGUF lacking valid panel scores.
  • Qualification must name the context it qualifies. Step 4 needs a claimed maximum occupied context, ≥4k generation headroom, the defined cold-cache resets, and post-run lab gates for each qualification run. “Agentic-usable” additionally requires ≥128k occupied tokens. Count ≥100 tool calls across screening, qualification, and any extra runs, as specified; publish truncations and agent-format errors too.
  • Make the search bounded and the stop rule meaningful. Predeclare which metric determines each step’s “gain”: official lab prefill for batching, minimum rolling-window decode at matched occupancy for quantization. A gain in one metric cannot excuse losing another acceptance criterion. Keep the phase’s 2–3-day timebox; disk sequencing alone should not defer x16 indefinitely.
  • Concrete revised decision: restore pinned IQ4_XS; run the two explicit batching changes; establish the context-dependent slowdown; screen IQ3_M, then IQ3_XXS only if justified. Run full stability qualification only on a speed-passing configuration. Unless the existing accepted scoring path establishes the kit band, publish the route as experiment-only, preserve its records, delete rejected weights under §9, and proceed to x16. Every 3-card record must include GPU UUIDs and display: off.

Runs & Data › Decisions

0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)

decisions/0009-no-x16-session.md

0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)

Operator, 2026-10-04 19:15 AEST: skip the x16 test; the expected difference is judged marginal. T12 (card move) and T05b are cancelled. The operator's call stands; the estimate it overrides is recorded for the reader: 0xSero's reference gives +17.9 % decode for 14–16 → ~26 GB/s, and our x8 baseline (lab C1 49–55, kit C1 46.9, agentic 43, floor not held) sits just under the 50 tok/s target, so reaching it now depends on tuning at x8.

Re-plan (single card = GPU2, CPU-lane x8, 13.5 GB/s; GPU1 measured identical)

  1. T13 PR #144 rerun at x8 against image f77c72f6 (lab, kit sweep with early exit off, TTFT, panel, 10-min run). Proof for 0xSero is "x8 on this box"; whether he accepts it is asked in Phase 3.
  2. T15 brought forward: the main recipe on EXL3 2.05 and 4.05 bpw, only the weights changed. At x8 the expert bytes per token are the bottleneck, so 2.05 bpw is the most likely route to ≥ 50 sustained; the kit panel decides whether it stays in the quality band (required for the headline and registry PR). 4.05 bpw is the quality anchor.
  3. T23 GLM exact mode on GPU2 at x8 (route test + quality reference; may not load: ~119 GB pinned on ~125 GiB).
  4. T14a/T14b grid on the best quant from 1–2 (expert cache GB × context × KV format; then prefill chunk and n-gram tier), designed as its own decision once 1–3 are in. T16 qualification and headline follow on the winner.
  5. GLM IQ2_M x16 rerun dropped; IQ3_XXS kept as the fallback reference.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Recorded as a deliberate restriction of the search, not evidence that x16 is marginal. Gap to target at x8: kit C1 46.9 → 50 needs +6.6 %; agentic 43 → 50 needs +16.3 %; there is currently no qualifying Flash-Next result.
  • Order: T13 at 3.05 → matched 2.05 / 4.05 screens (same launch, same context) → kit panel as a filter (top-1 ≥ 0.987, KL ≤ 0.0013) before any headline time → bounded tuning of in-band candidates, 3.05 included. Out-of-band results are published with kit eligibility, never as the registry path. "Bytes per token" is a hypothesis; transfer/cache evidence is recorded where the engine exposes it.
  • Acceptance unchanged: every eligible rolling 60-s window ≥ 50 and lab prefill > 1000 on the same config at the claimed occupied context. If no x8 config meets speed and quality together, that outcome is published as is.
  • T23 at x8: exact mode may be the GLM quality reference even below 15 tok/s (not a recipe); peak host memory and any load failure recorded (~110.8 GiB pinned vs ~125 GiB MemTotal). If it cannot load, the reference is the highest-bit GGUF that actually fits one card here, determined by a load test, not assumed to be IQ3_XXS.
  • Timebox and the 3 × < 3 % stop rule stay. Cancelled tickets (T12, T05b, x16 parts of T13/T14/T23) are recorded as skipped with the operator's reason.
Independent review (0009-no-x16-session.review.md)
  • AMEND. Respect the operator’s cancellation of x16, but describe it as a deliberate restriction of the search—not an evidence-backed conclusion that the gain is marginal. The strongest counter-argument is that the cited 17.9% uplift could change eligibility, whereas lower quantization risks losing the required quality band.
  • The target gap is material and measurable. Kit C1 46.9 → 50 requires +6.6%; agentic average 43 → 50 requires +16.3%, and reaching that average would still not prove the rolling floor. Applying the reference uplift illustratively gives 55.3 and 50.7 tok/s, respectively. These are scenarios, not predictions for this machine or workload.
  • Retain 3.05 bpw as a tuning candidate. The proposed sequence risks selecting an out-of-band 2.05 model or a slower 4.05 model before exploring the original quant. Concrete alternative: T13 at 3.05 → matched 2.05/4.05 screens → quality filter → bounded tuning of qualifying candidates, including 3.05.
  • Treat “expert bytes per token are the bottleneck” as a hypothesis. Neither the bandwidth measurement nor the quoted speeds establishes how much decode time is exposed transfer, compute, synchronization, or cache misses. Moving 3.05 → 2.05 bpw reduces nominal weight bits by 32.8%; under a purely weight-transfer-limited model that suggests about 49% more throughput. Metadata, cache residency and other costs can substantially change that result. Record transfer/cache evidence where available before assigning causality.
  • Use the panel early as a search filter. Require top-1 ≥ 0.987 and KL ≤ 0.0013 before spending headline qualification time on a quant. Publish outside-band results and retain their kit eligibility, but do not make them the sole path toward the registry target. Keep occupied context and other settings matched for the initial weight comparisons.
  • Preserve the actual acceptance test. Lab C1 over 30 seconds, a kit score, and a whole-run agentic average cannot establish a sustained floor. The winner needs every eligible rolling 60-second window ≥ 50, all qualification requirements, and lab prefill > 1000 on that same config, at the claimed occupied context. The existing “floor not held” evidence means there is currently no qualifying Flash-Next result.
  • T23 is useful at x8, but distinguish reference eligibility from recipe eligibility. Exact mode may serve as the GLM quality reference even below 15 tok/s; it then remains ineligible for a registry recipe. 119 decimal GB ≈ 110.8 GiB, leaving roughly 14.2 GiB against the stated 125 GiB MemTotal before runtime and system costs. That is plausible but unproven headroom; record peak host memory and any load failure.
  • IQ3_XXS cannot automatically become the GLM quality reference. The spec requires the highest-bit GGUF that fits when exact mode cannot load. Keep IQ3_XXS as a performance fallback, but separately determine the fitting quality reference and run the identical-task, identical-budget, uncapped comparison required by §5.
  • Make the amendment auditable and bounded. Update §6, §11 and the x8 risk discussion; record cancelled tests as skipped with the operator’s reason. Keep the phase timebox and diminishing-return stop rule. If no x8 config meets speed and quality together, publish that outcome explicitly rather than relaxing acceptance. The revised deliverable is the best qualifying result found within the x8 search.

Runs & Data › Decisions

0010 — CPU boost off for the rest of the campaign

decisions/0010-cpu-boost-off.md

0010 — CPU boost off for the rest of the campaign

Evidence (2026-10-04 19:48 AEST): hard reset during the T13 Flash-Next SGLang model load with core 2 already offline. The next boot decoded CPU:1 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], "uncorrected error caused a data fabric sync flood" — the same bank, status and syndrome as the core-2 faults (0004), now on core 1. Tally of fatal resets: 5, of which 3 during a Flash-Next SGLang load (expert repacking: heavy multi-threaded host work) and 2 after long uptime. Temperatures at idle are normal (Tctl ~56–60 °C). A second core failing the same way points to the CPU (or its voltage under boost transients) rather than one bad core.

Decision: disable Precision Boost from Linux (/sys/devices/system/cpu/cpufreq/boost = 0, all cores capped at the 3.4 GHz base clock, lower voltage), applied at boot by local-ai-bench-quarantine.service and immediately on 2026-10-04 20:12. Core 2 stays offline. Every record now carries host.cpu_boost and host.cpu_max_mhz.

Consequence: CPU-bound work (GLM experts on CPU, model loads) gets slower; PCIe-bound trellis decode should not. Records before 20:12 ran with boost on and are labelled by that field's absence (= boost on). If faults continue with boost off, the CPU is not usable for unattended runs and the remaining work pauses for the operator (RMA).

Runs & Data › Decisions

0011 — Flash-Next tuning at x8 (T14): grow the expert cache

decisions/0011-flashnext-x8-tuning-grid.md

0011 — Flash-Next tuning at x8 (T14): grow the expert cache

Evidence

  • T15 is not runnable: the offload engine's HostExpertStore only supports 3-bit experts (offload_store.py: if bits != 3: raise NotImplementedError), so 2.05 and 4.05 bpw crash at load in both images. Only 3.05 bpw remains (the CPU-expert fallback, SGLANG_EXL3_MOE_OFFLOAD=cpu, is far slower and not pursued).
  • T13 (PR #144 launch, f77c72f6, x8, boost off): lab C1 54.5–55.6, lab prefill 2,433–2,596, kit C1 48.2 (32k: 49.9), kit panel in band (top-1 0.9887, KL 0.00097), agentic whole-run 44.1, min 60-s window 37.9 (floor 50 fails).
  • Agentic per-request decode is flat across context (median 43.7 / 43.8 / 41.7 tok/s at <8k / 8–20k / 20–40k), so context length is not the limiter; the gap to lab C1 (55) is consistent with expert-cache misses on coding traffic (each miss streams experts over x8 PCIe). Hypothesis, to be tested by changing the cache size.
  • VRAM budget at ready (server log): expert cache 11.08 GB (auto-fit: free 19.82 − staging 0.48 − reserve 8.26), KV pool 5-bit 1.77 GB for 210k tokens, 3.23 GB still free after CUDA graphs. Context length is therefore a weak lever (the whole KV pool is 1.77 GB); the reserve is the strong one.

Grid (base = PR #144 launch on the latest main-built image a21efea6, GPU2 x8; one variable per step)

StepChangeExpected expert cache
R1SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB 8.26 → 6.0~13.3 GB (+20 %)
R2reserve → 5.0 (only if R1 is stable and gains ≥ 3 %)~14.3 GB (+29 %)
N1SGLANG_EXL3_NGRAM_TIER nvme → pinned (32.6 GB pinned RAM) on the best Rn-gram reads from RAM
G1--max-running-requests 1 --cuda-graph-max-bs-decode 1 (single-stream agentic) on the best, then re-fit the reservefrees graph memory

Metric per step: minimum rolling 60-s window and median per-request decode in the 10-min agentic run (the floor), with lab C1 and lab prefill checked for regressions. Every step runs lab,ttft,stab; the kit panel is re-run on the winner only (cache size does not change the math). Stop rule: 3 consecutive steps < 3 % on the floor metric. If no step lifts the minimum window to ≥ 50, Phase 1 closes with "not reached at x8", publishing the best config.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Cache misses are a hypothesis: the floor needs +31.9 % (26.4 → 20.0 ms/token); a 20–29 % larger cache need not deliver that. Expert-cache hit/miss and transfer stats are recorded if the engine logs them, else marked unavailable.
  • Order: R0 (unchanged PR #144 launch on a21efea6) → R1 reserve 6.0 → G1 single-stream graphs on R1 → reserve refit only if peak VRAM (prefill and max-occupancy) shows room → N1 pinned n-grams with host-RAM headroom recorded. R2 (5.0) is not run blind: measured headroom after R1 would be ~0.97 GB, after R2 ~−0.03 GB.
  • max-running-requests 1 is a recipe concurrency constraint, stated as such.
  • Floor = minimum rolling 60-s generation-only window after the first 60 s (harness definition); window coverage reported. One 10-min run screens; the winner then needs full §5 qualification, kit band, and ≥ 128k occupied for "agentic-usable". Closure wording: "50 tok/s floor not reached in the tested x8 configurations".

R0 result and re-plan (2026-10-04 22:42 AEST)

R0 (a21efea6, PR #144 launch unchanged): load 260 s, lab C1 55.8, lab prefill 2,419, kit C1 47.4 (32k 48.6), TTFT 2,009 / 2,681, panel in band (top-1 0.9897, KL 0.00097) — then the 10-min agentic run crashed: CUDA OOM. After serving starts, Triton kernels load lazily (log: "device-loaded after serving started … free 0.83 → 0.09 GiB") and consume the 3.23 GB that looked free at ready. So the 8.26 GB reserve is needed on this image; shrinking it cannot grow the expert cache safely. (f77c72f6 survived the same agentic run at the same reserve.) Re-plan: R1 (6.0) is already running and is kept as evidence; R1+G1 dropped. Next: R0 repeat (is the OOM reproducible?), G1 at reserve 8.26 (single-stream frees graph and request memory → headroom), N1 pinned n-grams.

  • R1 (reserve 6.0, 22:42): load OK, then CUDA OOM at the first lab request — reserve cannot go below 8.26.
  • R0 repeat, agentic run only (22:58–23:22): no OOM; 71 tool calls, 0 parse failures, whole-run 43.3, min window 39.7; lab after 57.4 / 2,537. The first R0's OOM is intermittent (it followed lab, kit sweep, TTFT and panel on the same server). Reported as an a21efea6 stability risk for PR #144; frequency unknown from 2 runs.
  • G1 single-stream at reserve 8.26 (23:22): min window 40.1 (+1 % vs R0 repeat), whole-run 43.1, lab 54.6–55.6.
  • N1 pinned n-grams (23:48): min window 40.9 (+3.1 %), whole-run 44.6, lab C1 53.7 / 51.3 (lower), prefill unchanged.

Closure (2026-10-05 00:11 AEST)

Stop rule met: R1 no gain (OOM), G1 +1 %, N1 +3 % (within run-to-run spread: two R0 runs and T13 span 37.9–39.7). Phase 1 result: the 50 tok/s floor was not reached in the tested x8 configurations. Best measured agentic floor ~40–41 tok/s (−18 to −20 %); lab C1 55–57, lab prefill ~2,400–2,600, kit panel in band. Best config for publication: the PR #144 launch (image a21efea6, or f77c72f6 which had no OOM in its run). No T16 finalist; T18 registry recipe not prepared; the PR #144 rerun evidence and the kit submission (with this tuning table) are prepared locally for Phase 3.

Independent review (0011-flashnext-x8-tuning-grid.review.md)
  • AMEND — growing the cache is a reasonable experiment, but the evidence does not establish cache misses as the bottleneck. Flat decode across context only weakens the context hypothesis. The floor needs +31.9% (37.9 → 50 tok/s); a 20–29% larger cache does not imply that speedup. The strongest counter-argument is that coding traffic accesses a broad expert set, so extra capacity may barely reduce misses.
  • Establish R0 on the new image first. T13 used f77c72f6; the grid uses a21efea6. Comparing directly changes both engine and reserve. Run the unchanged launch on the new, full digest-pinned image, then compare every tuning step against a baseline on that image.
  • R2 appears to exceed the measured memory budget. Assuming unchanged allocations and consistent units, R1 consumes another 2.26 GB, leaving approximately 0.97 GB after graphs; R2 consumes 3.26 GB, leaving approximately −0.03 GB. “Stable R1” cannot establish that R2 fits. Record peak VRAM during prefill and the claimed occupied-context run before reducing the reserve further.
  • Concrete alternative: R0 → guarded R1 → G1 → reserve refit → N1. Move graph reduction ahead of R2 when memory is tight, and measure each change separately. Treat max-running-requests=1 as an explicit recipe concurrency constraint; verify that the representative agentic workload fits it. For pinned n-grams, record host-memory headroom and end-to-end performance.
  • Test the mechanism alongside speed. Record expert-cache hit/miss counts, transferred bytes and transfer stalls if exposed; otherwise mark them unavailable and keep the attribution hypothetical. Going from 37.9 to 50 requires reducing generation time from 26.4 to 20.0 ms/token—a 6.4 ms/token saving. Establish whether expert transfers plausibly account for that much time.
  • Specify the floor calculation exactly. Ignore the first 60 seconds and calculate rolling windows using generation time only, excluding prefill, tools and idle time. Report window coverage and generation duration. Median request speed and whole-run average are diagnostics; neither establishes the floor.
  • Separate screening from acceptance. One 10-minute run can select candidates. The winner still needs the three §5 qualification runs—including fully cold and claimed maximum occupied context—≥100 tool calls across its runs, zero parse failures, repetition checks, all six lab gates afterward, and the kit quality band. “Agentic-usable” additionally requires ≥128k occupied tokens.
  • Narrow the closure claim. Close under the phase timebox or the specified three consecutive measured sub-3% gains, recording skipped branches explicitly. Publish “50 tok/s floor not reached in the tested x8 configurations” rather than implying x8 is exhausted. Prepare results and submissions locally; public publication remains Phase 3.

Runs & Data › Decisions

0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)

decisions/0012-glm-outcome.md

0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)

Result: no GLM-5.3-Flash configuration on this box reaches the registry's 15 tok/s gate; every route is a documented experiment, and no GLM registry PR is prepared. All numbers: lab C1 decode / lab prefill, current layout.

RouteCardsDecodePrefillToolsClassification
llama.cpp b11312 IQ2_M, ub409619.6288pass, 0 parse failuresexperiment (1 card < 15)
llama.cpp IQ2_M, ub2048, GPU2-first311.8218passexperiment (3-card, < 15)
llama.cpp IQ3_XXS / IQ4_XS310.4 / 8.3132 / 156passrejected for speed, quality unmeasured
llama.cpp Q2_K / IQ3_XXS19.2 / 8.378 / 65passrejected for speed
ik_llama 5bf8f0fe IQ2_M (no MTP)313.4–14.0185fails (tool calls unparsed, 0005)experiment-only, unattested image
ik_llama IQ2_M + MTP1 / 38.2–12.6~150–166failsexperiment-only; OOM / server crash
0xSero glm53-flash-offload, exact mode (x8)17.4–7.5720–739pass, 0 parse failuresquality reference; route below gate at x8
  • T25a: no current-layout finalist (decision 0007; rolling-floor failures measured in every 10-min run).
  • T24: the uncapped candidate-vs-reference comparison runs only for finalists, so it is not run. Exact mode loads on this box (~120 GB host RAM, 23.3 GB VRAM) and stays available as the reference if a future config qualifies.
  • T25b: 1-card routes < 15 → experiment; 3-card routes < 15 → no separate recipe; ik_llama → experiment-only until a tool-parser fix and an attested local-ai-images build (operator to raise with 0xSero).
  • T26: not applicable.
  • Hardware context for the reader: dual-channel DDR4-3200 (~16.6 GB/s host memcpy measured), x8 PCIe (13.5 GB/s), 15 cores at base clock (decisions 0004, 0010). Fast mode (0xSero's 28.9 tok/s route) needs ~238 GB RAM: not possible on AM4.

Runs & Data › Decisions

0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)

decisions/0013-flashnext-oom-fix.md

0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)

Operator request (2026-10-05): rerun the Qwen3.8 (Flash-Next) test and fix the stability issues.

Diagnosis (R0 first run, 2026-10-04 12:32 UTC)

  • The server died in a prefill: offload_runtime._prefill_parts_compact → `torch.AcceleratorError: CUDA error: unknown error, 29 s after Triton kernel '_row_logsumexp_kernel' device-loaded after serving started (free device mem: 0.00 GiB)`. Earlier in the same lifetime, during the 166k-token context gate, PyTorch logged repeated expandable_segments: memory mapping failed with OOM (free 10 MB of 25.3 GB).
  • At ready the engine reports 3.23 GB free, but kernels that load lazily after serving starts, plus prefill activations for 12,288-token chunks, consume it. Whether a run crashes depends on which kernels load late and the size of the next prefill: 1 of 3 lifetimes of the unchanged launch crashed; reserve 6.0 crashed at once.

Plan (GPU2 x8, full flow lab → kit sweep → TTFT → panel → 10-min agentic, the sequence that crashed)

  • Every flow now samples VRAM every 0.5 s (logs/vram/*.csv, summary = minimum free MiB): headroom is measured directly instead of waiting for random crashes.
  • Runs: baseline (unchanged launch) → A reserve 8.26 → 9.26 GB → B --chunked-prefill-size 12288 → 8192.
  • Pick the fix with the larger minimum free VRAM and the smaller speed cost (lab C1, kit C1, prefill, agentic floor), then repeat it twice (3 clean lifetimes) before calling it fixed. If neither leaves ≥ 1 GB minimum free, combine them.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Framed as memory-pressure mitigation, not proof the intermittent failure is gone: 3 clean lifetimes would still occur with probability 29.6 % at the old 1-in-3 crash rate (95 % upper bound on crash probability 63 %). The main evidence is the measured VRAM headroom margin; the diagnosis stays a hypothesis (the final unknown error is not itself a proven OOM). Allocator peak/reserved stats are not exposed outside the engine → recorded as unavailable.
  • Configs, one variable each: baseline (reserve 8.26, chunk 12,288), A (9.26, 12,288), B (8.26, 8,192); (9.26, 8,192) only if neither single change leaves margin. Actual expert-cache GB recorded from the server log for each.
  • Selection: Flash-Next has no qualifying config at x8 (floor not reached, decision 0011), so the target-based rule cannot pick a winner; the rule is **the config with the largest minimum free VRAM whose lab C1, kit C1, lab prefill and agentic floor stay within 3 % of baseline**; ties → baseline settings. ≥ 1 GB minimum free is an engineering margin, not a spec criterion.
  • Image provenance: a21efea6 = ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163, local-ai-images run 37113268282 on main, trellis-serve b1e74af; it replaces the PR's branch-built digest per 0xSero's PR #144 comment of 2026-10-03.
  • Note: the queue skipped the baseline line (identical text already in campaign.done from 2026-10-04); re-added with an explicit step list, so it runs after A and B.

Results (2026-10-05, GPU2 x8, same full sequence lab → kit sweep → TTFT → panel → 10-min agentic)

ConfigMin free VRAM (0.5 s samples)Lab C1Kit C1Lab prefillAgentic floor / avgOutcome
baseline (8.26, 12,288)2 MiB (10:41, kit sweep)49.947.82,419— / 43.0crash in agentic run, 6 tool calls
A (9.26, 12,288)912 MiB48.1–49.743.72,408–2,56437.4 / 41.1clean, 71 tool calls
B (8.26, 8,192)1,060 MiB49.5–54.948.22,131–2,23739.5 / 43.9clean, 64 tool calls

Baseline crash: Triton kernels loaded after serving started at "free device mem 0.00 GiB" (alloc_extend_kernel, _row_logsumexp_kernel, _sparse_gqa_chunk_prefill), then CUBLAS_STATUS_INTERNAL_ERROR in cublasGemmEx (cuBLAS could not get workspace). The unchanged launch has now crashed in 2 of 4 lifetimes; it runs the card to ~0 free VRAM. Chosen: B — the most headroom, decode unchanged (kit C1 +1 %, floor −0.5 %), prefill −8 to −12 % (still ~2,100+, over twice the 1,000 target). A costs 8–13 % decode (smaller expert cache). Two more B lifetimes queued.

Confirmation (2026-10-05 10:59–12:29 AEST): fix B, 3 lifetimes

LifetimeStep orderMin free VRAMLab C1 (before / after)Kit C1Agentic floor / avg, tool callsOutcome
B1lab, kit, ttft, panel, stab1,060 MiB54.9 / 49.548.239.5 / 43.9, 64clean
B2lab, kit, ttft, panel, stab1,060 MiB56.4 / 53.547.038.8 / 43.2, 62clean
B3lab, ttft, panel, kit, stab1,060 MiB55.0 / 45.048.140.4 / 44.0, 61clean

Result: memory-pressure mitigation confirmed for this sequence. The minimum free VRAM is identical in all three lifetimes (1,060 MiB), so the peak is deterministic and the margin is real, against ~0 MiB and a 2-in-4 crash rate for the unchanged launch. 187 tool calls, 0 parse failures across B1–B3. Cost: lab prefill −8 to −12 % (2,131–2,240). Lab C1 varies 45–56 between 30-s windows on every config (single-sample noise), kit C1 is stable at 47–48. Recommendation for PR #144 / the registry launch on 24 GB cards: --chunked-prefill-size 8192. The agentic 50 tok/s floor is a separate, unchanged result (decision 0011): ~39–40 tok/s at x8.

Independent review (0013-flashnext-oom-fix.review.md)
  • AMEND. The two interventions are reasonable, but the plan can declare “fixed” without demonstrating either the cause or §5 qualification. Treat this as mitigation testing; select the fastest qualifying config, with headroom as supporting evidence.
  • Strongest counter-argument: three clean lifetimes provide weak evidence of reliability. If the true crash probability remains 1/3, three successes still occur with probability 29.6%. With zero failures in three independent trials, the one-sided 95% upper bound on crash probability is 63.2%. Three runs satisfy the spec’s qualification count only when they meet its prescribed conditions; they do not establish that intermittent failure is eliminated.
  • The diagnosis should remain a hypothesis. Mapping OOMs and exhausted VRAM strongly support memory pressure, but CUDA error: unknown error does not establish the final failure’s cause. A kernel-loading message 29 seconds earlier is correlation. Record the failing allocation/request and available allocator diagnostics; describe the outcome as “memory-pressure mitigation qualified” unless causal evidence improves.
  • A 0.5-second VRAM sample is insufficient to certify peak headroom. It can miss allocation bursts, and device-free memory alone cannot distinguish reusable allocator reservations from live allocations. Supplement sampling with per-stage allocator peak allocated/reserved memory, allocation-failure records, and request occupancy/chunk size. Keep collection outside the engine and write logs to tmpfs as §9 requires.
  • Make the comparisons unambiguous. Use baseline (reserve 8.26, chunk 12,288), A (9.26, 12,288), and B (8.26, 8,192). Explicitly reset reserve for B. Test (9.26, 8,192) afterward if necessary, comparing it against either single-change config. This preserves §10’s one-variable-family rule.
  • Neither adjustment has a guaranteed memory benefit of the stated size. Raising reserve by 1.00 GB may reduce expert-cache capacity; reducing chunk size by 33.3% reduces only chunk-dependent allocations, not total peak usage. Standardize GB/GiB/MiB, record actual cache capacity, and justify the ≥1 GB threshold against observed peaks. It is an engineering margin, not a spec acceptance criterion.
  • Replace “larger minimum free VRAM and smaller speed cost” with an explicit selection rule. Require lab decode ≥50 tok/s, lab prefill >1000 tok/s, the quality band, and §5 on the same config at the claimed occupancy. Then choose the fastest qualifying config within the search budget. The cited 53.2 tok/s reference permits only about 6.0% decode loss before missing the target; this machine’s measured baseline determines its actual margin.
  • Make the three finalist runs conform explicitly to §5: cold with all specified caches reset, claimed maximum occupied context, and repeat; zero OOM/crash/parse failures, ≥100 tool calls across the config’s runs, repetition checks, the rolling decode floor, and all six lab gates after each run. A 166k context gate alone does not demonstrate agentic qualification there.
  • Resolve provenance before running. a21efea6 is insufficient identification, and it differs from the PR #144 digest specified in Phase 1. Record the full digest, source commit, attestation/allowlist eligibility, and reason for replacing the specified image. Store each immutable launch before execution and publish every failed attempt with explicit unavailable metrics.

Runs & Data › Decisions

0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)

decisions/0014-flashnext-vs-27b.md

0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)

Question: on this box, which is the better local model for agentic coding (and generally): Flash-Next (180B MoE, 6B active, experts streamed from RAM) or the dense 27B the operator runs daily?

Configs (both on GPU2, PCIe 4.0 x8, 15 cores, boost off; one card each)

  • Flash-Next: PR #144 Phase D launch on a21efea6 with the stability fix (--chunked-prefill-size 8192, decision 0013), EXL3 3.05 bpw, 204,800 context, up to 4 concurrent requests.
  • 27B: the operator's daily server exactly (omarchy-local-ai recipe qwen38-27b-exl3-3bpw-rtx3090-sglang-tp1): ghcr.io/0xsero/sglang-exl3@sha256:84f75f34…, EXL3 3.00 bpw (turboderp/Qwen3.8-27B-exl3@6fe61ad6…), NEXTN/MTP speculative decoding (3 steps, 4 draft tokens), 204,800 context, single stream.
  • Each is the best known recipe for its model on a 24 GB 3090; they differ in more than one variable by design (this compares models as deployed, not an isolated knob). Sampling, task set, token budget and harness are identical.

Measurements (same order for both; one server lifetime each)

AreaStepMetric
Speedlab (registry gates), kit sweep, ttftdecode C1 (C2–C4 where supported), prefill 8k–64k, TTFT
Long contextneedle (kit method, thinking off)retrieval at 128k (depths 0.2/0.5/0.8) and ~192k (0.5)
Agentic reliabilitystab (10-min agentic, 8-task §5 set)tool calls, parse failures, loops, decode floor
Agentic codingqeval (T07, uncapped, one pass) on tasks-compare30solve rate (official SWE-bench eval), tokens and wall time per task
Footprintload + VRAM samplerload time, host RAM, VRAM, min free VRAM

tasks-compare30: 30 SWE-bench-Lite test instances drawn with random.Random(20261005).sample (uniform, not hand-picked), images pinned by digest after pull; same 60k completion-token budget per task for both models. With n = 30, a difference in solve rate is reported with a paired comparison (tasks solved by one and not the other) and an exact McNemar p-value; small differences are labelled inconclusive rather than ranked.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Labelled an exploratory deployment comparison on this box (the §10 one-variable rule is waived for it, as the two recipes differ in engine build, MTP and concurrency by design). Conclusions are per priority — agentic coding success, interactive speed, long context, footprint — never "generally better".
  • compare30 is exploratory: exact McNemar on 30 single attempts needs ≥ 6–0 discordant tasks for p < 0.05 (8–2, a 20-point gap, gives p ≈ 0.11). The site publishes the paired per-task table, the solve-rate difference with a 95 % CI and the p-value; small differences stay unranked.
  • Cache state: the flow serves cold (container restart, page cache dropped) and then measures in a fixed order, so qeval runs warm for both; the 10-min stab run is screening, not §5 qualification. Flash-Next's kit band is already in band for this config (decision 0013: top-1 0.989, KL 0.00092); the 27B has no kit panel (different model), so its quality evidence is the solve rate only.
  • Budget: up to 60k completion tokens per task (all of a task's model calls) → worst case 1.8M tokens per model; early submits cut this sharply. Infrastructure failures (image, evaluator) are reported separately from unsolved. Accepted tokens/s and acceptance rate are recorded for the 27B's MTP.

Finding during the 27B run (2026-10-05 18:07 AEST)

The operator's 27B server advertises --context-length 204800, but its KV pool is max_total_num_tokens=135424 (mem-fraction 0.80, fp8 KV, MTP draft buffers). The registry lab's context gate (0.85 × 204,800 = 166,667 tokens) is rejected: "Input length (166667 tokens) exceeds the maximum allowed length (135418 tokens)". So the 27B as deployed really holds ~135k tokens, not 204,800 — the same advertised-vs-real gap seen in the operator's pi session. The comparison reports both; an extra lab run uses the identical server with harness context 131,072 (…-omarchy-ctx131k launch) so its speed gates are measured. Flash-Next's KV pool is 209,984 tokens (fits 204,800).

Independent review (0014-flashnext-vs-27b.review.md)
  • AMEND. Comparing the best deployed recipes is useful, and differing engines/MTP settings are legitimate for that question. The strongest objection is inferential: 30 tasks, one attempt each, and one server lifetime cannot reliably establish which model is better “generally.” Frame this as a deployment comparison on this machine, with separate conclusions for coding quality, latency, throughput and resource use.
  • Thirty tasks can miss substantial differences. With exact two-sided McNemar, 5 wins versus 0 losses gives p = 0.0625; 6 versus 0 gives p = 0.03125. Even 8 versus 2—a 20 percentage-point solve-rate advantage—gives p ≈ 0.1094. Publish the paired outcome table and a confidence interval for the difference. Keep compare30 exploratory; if a definitive ranking matters, predeclare a larger fixed sample before examining results.
  • One stab run is screening, not §5 qualification. Unless this exact Flash-Next config already has qualifying evidence, require the three qualification runs: cold, claimed maximum occupied context, and repeat; ≥100 aggregate tool calls; the rolling decode-floor measurement; and post-run lab gates. Otherwise label it experiment-only and its stability unqualified. “One server lifetime” needs an explicit exception for the required cold restart.
  • The measurement order systematically warms Flash-Next’s streamed experts. lab → kit → ttft → needle → stab → qeval measures coding after extensive preparation, whereas cold agent startup may behave differently. Keep the warm deployment comparison, but add the specified cold qualification and cold first-task latency. Disable prefix caching and use fresh prompts for TTFT; record expert, n-gram and OS cache states separately.
  • Configured context and needle retrieval do not establish agentic usability. A ~192k needle pass supports retrieval at that occupancy, not reliable coding there. Report server-counted occupied tokens and ≥4k generation headroom. Award “agentic-usable” only after §5 qualification at ≥128k occupied tokens; otherwise say “configured for 204,800.”
  • Restore the explicit Flash-Next quality gate. Require or cite matching evidence for top-1 ≥0.987 and KL ≤0.0013 against the pinned reference panel before any spec headline or registry claim. SWE-bench solve rate complements that requirement; it does not replace it. Publish outside-band results with the prescribed label.
  • Make the deployment exception reproducible. Explicitly amend §10’s “one variable family” rule for this comparison and avoid attributing differences to architecture alone. Store full image digests, 40-character weight revisions, file hashes, immutable launches, GPU UUID, harness/evaluator commits, ordered task IDs and the source dataset revision. Record accepted tokens/s, MTP acceptance rate and draft settings as §4 requires.
  • Account for cost and stochastic variation. The completion allowance totals 3.6 million tokens across both models—20 hours of generation at 50 tok/s, before prefill, tools and evaluation. Reserve that time within the phase budget. Define which tokens count toward 60k, distinguish infrastructure failures from model failures, and report total wall time and tokens per successful solve alongside per-task results. A single attempt measures this run’s outcomes; paired statistics do not eliminate sampling variability.
  • Concrete alternative: run compare30 as a clearly labelled exploratory deployment comparison; reuse documented qualification evidence where it exactly matches, complete missing qualification/cache checks, and publish a tradeoff table. Recommend a model only for a stated priority—such as coding success or interactive latency. Remove “and generally,” and leave small or uncertain quality differences unranked.

Runs & Data › Decisions

0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)

decisions/0015-next-tests-plan.md

0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)

What the thread says

  • Operator showed local-ai-bench.org/compare and the SWE-bench-Lite contamination finding; asked "Shall I create a benchmark from recent github issues?" — Sero: "Yes". Operator: "land a novel benchmark, and make it real", and later "an agent/coding benchmark for multiple languages without disclosing the questions/content".
  • Sero (funding): keep running experiments, turn them into cool demos (Omarchy + the local-ai plugin, "wacky stuff") that he will amplify; fundraiser in a few weeks with 3 goals (small/medium/big). Hardware advice: stick with NVIDIA (FLOPs, CUDA) — vs the operator's ideas (Epyc 256 GB + 4th 3090, or an M5 Ultra).
  • Operator wants to go deep on Qwen3.8-Flash-Next ("Qwen 4 is around the corner").
  • Links: Niko1221/Strata (operator: "amazing decoding numbers … how much better than Qwen 3.8 27B?"); kacper-daftcode/vLLM-Moet + 0xSero/moetier (Sero: "give this to claude your rig will fly", "I don't use gguf"; operator: vLLM-Moet says Blackwell → Sero: moetier "shouldn't need Blackwell"). Operator: "Will test these through the benchmark."
  • Repo facts: Strata = C++ engine for Flash-Next GGUF quants (Q2_0 … IQ3_S, "Coder" IQ1_M); claims 94 tok/s on a 12 GB RTX 5070 and "100–140 tok/s" expected on a 3090. vLLM-Moet = hand-written SM120 (Blackwell) SASS kernels — cannot run on SM86 3090s. moetier = Sero's data-first standard + simulator for expert placement across VRAM/RAM/NVMe; its loop's primary target is GLM-5.3-Flash on 1× 3090 + 55 GB DDR4 + NVMe at 50 tok/s, with run records (registry/runs) carrying a standard table, utilization and chokepoints (moetier probe).

Plan (ordered; each item ends in published records)

  1. Tell Sero the PR #144 stability finding now (operator sends; draft in prs/pr144-proof.md): a21efea6 runs a 24 GB card to 2 MiB free and crashed 2/4; --chunked-prefill-size 8192 fixes it (3/3, decode unchanged). He is the maintainer, it is the cheapest high-value result we have.
  2. moetier on this box (Sero's explicit ask). (a) hardware record for omarchy-gpu (GPU2 x8 13.5 GB/s, 2-ch DDR4, 16.6 GB/s memcpy, 1 TB NVMe 3.4 GB/s rand) and a Flash-Next model record if missing; (b) moetier probe on the trellis Flash-Next config → utilization + chokepoints run record: name the binding constraint behind the ~40 tok/s agentic floor at x8; (c) reproduce his GLM-5.3 NVMe-mode recipe at a 55 GB RAM budget on our single NVMe (an independent hardware datapoint for his G1/G2 work); (d) moetier sim the fundraiser hardware goals.
  3. Strata vs 27B vs trellis Flash-Next (operator's question). Pinned Strata release; Flash-Next GGUF quants (Q2_0, IQ2_XS, IQ3_S, Coder) on GPU2, then 2-3 cards GPU2-first. Same suite as decision 0014 (lab, kit sweep, TTFT, needle, 10-min agentic) + quality vs the EXL3 3.05 reference (KLD/top-1 where the API exposes logprobs). Test the "100–140 tok/s on a 3090" claim. GGUF → operator-facing comparison, not a registry recipe (Sero).
  4. Fresh-issues benchmark (decontaminated, held out). Pilot: ~50 Python tasks from issue-linked PRs merged after 2026-09-01 (after both models' release), auto-built Docker envs (reuse SWE-rebench / SWE-bench-Live RepoLaunch tooling), validated FAIL_TO_PASS/PASS_TO_PASS with the gold patch, run through our harness. Held out: publish aggregate scores, repo list, task-ID hashes and a few retired samples; refresh monthly. Then add TypeScript, Go, Rust (multi-language, per the operator). Keep the gold-overlap check as a standing guard.
  5. Run the three contenders on it: 27B, trellis Flash-Next (fixed), best Strata quant; paired analysis.
  6. Demos for Sero / the fundraiser: Omarchy + local-ai plugin — (a) 27B as a fast coding agent (one-shot app), (b) Flash-Next answering over a ~190k-token codebase; short screen recordings + numbers from the records.
  7. Fundraiser evidence: per goal (small: 4th 3090 / 2nd NVMe; medium: Epyc 8-ch 256 GB + x16 lanes + 4th 3090; big: a Blackwell card → vLLM-Moet/moetier), projected tok/s from moetier sim calibrated on our measured runs.
  8. Qwen 4 day-0 readiness: launch templates + the fresh-issues set ready so a new model runs the same day.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Milestone 0 first: Phases 1–2 are closed as documented results (0011, 0012); their Phase 3 material (PR #144 evidence, kit submission) goes out before new work is presented as findings. PR #144 wording: "no crashes in 3 full-sequence runs with chunked prefill 8192 (vs 2 of 4 without); decode unchanged within ±2 % (kit C1 47–48); prefill −8 to −12 %" — a mitigation, not a proven fix.
  • moetier: answer one measurable question — what binds the ~40 tok/s agentic floor (bytes per token over PCIe, expert-cache misses, CPU lane, GPU idle) — with measured counters, then change one variable family. The GLM reproduction states its RAM budget in GiB including pinned memory, swap off, cache resets declared; 50 tok/s is Sero's hypothesis, and registry gates still apply.
  • Strata: screen quants on GPU2 only (one 10-min agentic run + speed suite each), qualify only finalists, and add 2–3 cards only for a finalist with a measured reason. Verify what "Coder" changes (weights/behaviour). Quality without logprobs = "unavailable", never in-band.
  • Fresh-issues set = "temporally held out", not "decontaminated": require issue and fix after 2026-09-01, overlap check against existing benchmarks, no network access for the agent during evaluation, tasks frozen before any tuning. A separate small development set is used for choosing configs; finalists run once on the held-out set. 50 tasks is a pilot (±14 pp at 50 %); report paired, repository-aware uncertainty.
  • Secrecy vs reproducibility: a private immutable manifest (commits, patches, tests, env digests, harness settings) with a published commitment hash; controlled reproduction on request; full artifacts released when a task set retires. Active-set raw logs stay out of the public site (SPEC §8 amended accordingly).
  • Fundraiser numbers are conditional projections with stated assumptions and sensitivity ranges, kept apart from measured results; topology (slots, lanes) is checked before any 4th-3090 projection.
Independent review (0015-next-tests-plan.review.md)
  • AMEND. The strongest objection is scope drift: the agreed deliverable is qualified Flash-Next and single-3090 GLM recipes; this plan adds a benchmark product, several engine comparisons, demos and hardware projections before closing those deliverables. Keep the new work, but put it behind an explicit milestone: publish the fastest qualifying configs—or documented failures within the spec’s timebox—and prepare their submissions.
  • Report PR #144 as a promising mitigation, not a proven fix. Moving from 2/4 crashes to 0/3 is useful evidence, but three successes still permit a per-run failure probability of 63% at a one-sided 95% confidence bound under independent, identical trials. Use “no crashes in three qualification runs,” verify the same occupied context and workloads, and quantify “decode unchanged.” Confirm Phase 3 publication requirements are already satisfied or explicitly amend them.
  • Make moetier answer a measurable bottleneck question. Going from ~40 to 50 tok/s requires 25% more throughput, or 20% less time per token. If PCIe alone were saturated at 13.5 GB/s, that implies reducing traffic from approximately 338 to 270 MB/token. These are conditional bounds, not a diagnosis. Measure transferred bytes, expert-cache misses, CPU work and GPU idle time, then change one variable family. A utilization report alone cannot establish causality.
  • Define the GLM reproduction before running it. Specify whether “55 GB” means decimal GB or GiB, and whether it includes pinned allocations, page cache and engine overhead; declare swap and cache-reset behavior. Treat 50 tok/s as the supplied recipe’s hypothesis, not an expectation. Registry eligibility still requires the ≥15 tok/s floor, uncapped quality evaluation and ≥80% of reference solves; reproducing a fast simulator or engine run does not satisfy those gates.
  • Separate Strata’s speed claim from its eligibility. The supplied 100–140 tok/s claim is 2–2.8× the Flash-Next target, but needs matching context, cache state, sampling and accepted-token accounting. A low-bit quant can win throughput and fail quality. Missing logprobs mean “quality unavailable,” which cannot establish the required top-1 ≥0.987 / KL ≤0.0013 band. Verify whether “Coder” changes weights or model behavior before presenting it as a quant-only comparison.
  • Rename the fresh set “temporally held out”; “decontaminated” overclaims. A PR merged after model release can fix an issue published long before release, and an agent with network access can retrieve the public solution. Require fresh issue/test provenance, inspect overlap with existing benchmarks, block solution retrieval during evaluation, and freeze evaluation tasks before tuning. These controls reduce contamination risk; they do not prove its absence.
  • Treat 50 tasks as a pilot, not a decisive leaderboard. At a 50% solve rate, 50 independent tasks give an approximate 95% margin of ±14 percentage points; one task changes the score by 2 points. Repository clustering can make uncertainty larger. Report paired outcomes and uncertainty based on discordant tasks, with repository-aware analysis. Use a development set for choosing the “best Strata quant,” then evaluate finalists once on the held-out set.
  • Resolve secrecy versus reproducibility explicitly. Public aggregate scores, repository names and task-ID hashes do not let another evaluator reproduce scores, and hashes of enumerable public IDs may reveal tasks. Keep a private, immutable manifest containing base commits, patches, tests, environments and harness settings; publish its commitment and arrange controlled independent reproduction. Amend §8 so public raw logs cannot accidentally expose active tasks, with full artifacts released upon retirement.
  • Make fundraiser projections conditional, with uncertainty. A fourth 3090 adds 24 GB nominal VRAM, not guaranteed usable bandwidth or speed; the actual slot, lane and chipset topology must be established first. A second NVMe helps only if measured storage contention is binding. EPYC and Blackwell projections require calibration beyond this AM4/SM86 machine. Publish assumptions and sensitivity ranges; keep measured results separate from simulated forecasts.
  • Concrete replacement order: narrow PR #144 report → close existing Flash-Next/GLM qualification and submission material → probe the measured bottleneck → screen Strata on GPU2 and qualify only finalists → freeze and validate the 50-task pilot → paired contender evaluation → demos and conditional fundraiser scenarios. Four quants across three layouts already imply 12 configs: up to eight hours of screening plus qualification alone, before loading, lab, sweep and quality checks. Reserve multi-card expansion for a single-card finalist with a demonstrated reason to benefit; label the ~190k demo by actual occupancy and qualification status.

Runs & Data › Decisions

0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")

decisions/0016-fresh-issues-stage2.md

0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")

Context: decision 0015 item 3 (temporally held-out Python set, issue AND PR created ≥ 2026-09-01, private tasks, published scores + commitment hash). Stage 1 (tools/fresh/, data in ~/dev/local-ai-exp/fresh-private/, never in this repo) gave a pool of 5,450 candidates and a ≤ 3-per-repo shortlist of 763 (275 repos; 249 easy / 271 medium / 243 hard; 528 from repos not in any public SWE dataset). Operator choice (2026-10-08): the setup/filter LLM is this Claude subscription (claude -p, Opus) — not an API key; Anthropic therefore sees which tasks we chose ("semi-private for Anthropic models", stated next to every score).

Plan

  1. Stage-2 input (deterministic, before any LLM): from the shortlist drop rows with base commit dated before 2026-08-24 (5), no .py test file (5), and repos created after 2025-09-01 or with < 10 distinct contributors in the last year (guards against single-author agent-generated issue→PR streams seen in the top repos). Sample 150 by difficulty (easy 45 % / medium 40 % / hard 15 %), seen/unseen repos ≈ 1:2. Seed fixed, list recorded.
  2. Clarity filter (Opus, one claude -p call per task, JSON out): the SWE-bench-Verified rubric — is the issue well specified, can a correct fix be judged by the hidden tests without guessing names/messages that only the tests define, is the fix given away in the issue. Reject scores ≥ 2 (0–3 scale) on either axis.
  3. Environment (deterministic skeleton + agent-written setup):
    • Repo snapshot: git init + git fetch --no-tags origin <base_commit> (ancestors only; no branches, tags or later objects exist in the image), checkout detached, remote removed, reflog expired, gc --prune=now; a post-build check asserts git rev-list --all contains no commit dated after the base and the gold blobs are absent.
    • Base image python:<ver>-bookworm pinned by digest. Opus (claude -p --model opus, Bash tool restricted to docker exec into a throwaway per-task container with network, 45-min cap, ≤ 60 turns) writes setup.sh (system packages, pip/uv installs) and the test command pytest -rA <test files>; installs pinned to versions available before the PR's creation date where the agent can (uv pip --exclude-newer). The final image is rebuilt from setup.sh without the agent (reproducible), tagged lab/fresh.<id>, digest recorded.
    • LLM-written log parsers are not used: pytest -rA + swebench parse_log_pytest only (no host-side code from the agent).
  4. Validation (no LLM): in the image, run the test files on base + test_patch (expect failures), then base + test_patch + gold patch; FAIL_TO_PASS = fail→pass, PASS_TO_PASS = pass→pass. Require ≥ 1 F2P, gold resolved 3/3 (flaky tests dropped from P2P, task dropped if an F2P is flaky), empty patch resolved 0/1, test run < 10 min. Rows emitted in our swebench 5.x format (eval_script, parse_log_pytest, version fresh-2026-09, image digest) into a private task dir; swebench evaluates them with network off for the agent under test.
  5. Freeze: from validated tasks, re-apply ≤ 3 per repo, stratify, pick 50 test + 10 dev (dev is for harness debugging only; never reported). Manifest (ids, base, patches, digests) committed only in fresh-private; publish sha256(manifest || salt) and the counts/distribution now, the salt and manifest when the slice is retired.
  6. Runs: Qwen3.8-27B (ctx131k), Flash-Next trellis c8192, Strata finalist; same harness as compare30 (uncapped, one pass), paired analysis (McNemar), gold-overlap per task as a leak check (flag > 50 % verbatim reproduction).

Scheduling and cost

Stage 2 is CPU/disk heavy (docker builds, test runs): it runs only as sh lines of the campaign queue, never during a GPU measurement. Parallelism 3 tasks. Disk: ~1–2 GB per image → ≤ 200 GB worst case before pruning failed builds. Subscription usage: ~150 filter calls + ≤ 150 setup sessions; stop and report if rate-limited.

Risks

Yield may land < 50 (published Python validation yield ~19 %): widen to the pool (reserves) and to October issues as a second slice rather than relaxing the date rule. A repo snapshot without later commits can still contain the fix if it was committed to the base before the PR (rare; the gold-overlap check at run time flags it).

Outcome after review (GPT-6.1 Sol: AMEND) — accepted

  • Supplementary pilot, contamination-reduced (not "uncontaminated"); labels next to every score: "Opus-curated", "semi-private for Anthropic models", rubric + Opus id + rejection counts published. Does not replace §5 qualification or the reference panels. Public release only after Phase 3 review.
  • Ordered batches, not 150: an ordered September list (stage-2 filters applied, stratified, seed fixed) is frozen now; attempts run in batches of 50 until 60 validated (expect ~320–380 attempts). Budgets: setup cap 30 min/task, ≤ 3 parallel, stage-2 wall time ≤ 40 h, image disk ≤ 150 GB (rejected images pruned immediately). If a budget runs out, stop with an undersized pilot and say so. October = separate slice.
  • Git invariant: image HEAD == base SHA; refs = HEAD only; git rev-list --all == ancestry of base; no stray working-tree files vs git status; gold-patch files are never mounted or copied into setup containers. Base must fail the hidden F2P tests (acceptance check).
  • Reproducibility: keep setup.sh, resolved pip freeze/uv lock with hashes, apt package versions, build logs, test command and image digest. Setup containers: CPU/RAM/PID limits, no host secrets, no Docker socket.
  • Validation: collected node IDs must match between runs; skips/xfails/collection errors counted explicitly; test patch must not disable tests; each repetition in a fresh container; gold 3/3 + empty 0 recorded with every exclusion and reason (3/3 is weak evidence on flakiness — reported as such).
  • Protocol frozen before candidate runs: manifest bytes, high-entropy salt, ordered reserves, selection rules, harness/parser SHAs, sampling, per-task token budget (same for all models; the compare30 harness settings), tool budget, context policy, retries, timeout treatment. Weight revisions recorded per model.
  • Reporting: one primary comparison predeclared — Strata finalist vs Qwen3.8-27B (the operator's question); others exploratory. Discordant counts, exact McNemar, repo-clustered intervals, ±14 pp margin stated at n = 50.
  • Gold overlap: descriptive audit only (denominator = gold added lines), never used to exclude or rescore.

Predeclared before any candidate run (2026-10-09 05:50 AEST)

  • Strata finalist = IQ3_S (launch strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb): T41 screen agentic floor 87.3 tok/s (Q2_0 123.3), compare30 29/30 (Q2_0 28/30); chosen as the higher-quality quant that still clears the 50 tok/s floor. Primary comparison: Strata IQ3_S vs Qwen3.8-27B (ctx131k launch), exact McNemar on the 50-task test slice. Exploratory: Flash-Next trellis c8192, Strata Q2_0.
  • Harness: tools/agentic/quality_eval.py at the commit recorded in each result, uncapped one pass, 60,000 completion tokens per task, sampling temperature 0.7 / top_p 0.95 / seed 1234 / max_tokens 8192 (compare30 settings), agent container network none.

Runs & Data › Decisions

0017 — Site structure: story-led home, findings, recipes, model cards

decisions/0017-site-structure.md

0017 — Site structure: story-led home, findings, recipes, model cards

Operator request (2026-10-10): pages were messy; home must show the sequence of what was done and a summary of achievements; every model gets a model card (link, details, learnings to run). Debated with GPT-6 Astra (Codex, 2 rounds) before building.

Outcome

  • Navigation: Home · Findings (speed, coding quality, GLM-5.3-Flash, hardware) · Recipes · Models · Method & Hardware · Runs & Data · Funding & Contributions. Old tabs (#charts, #compare, #fresh, leaderboards) kept for links, marked superseded or moved under Data.
  • Home: dated current answer (speed config and task-success config named separately), three bounded achievement summaries with conditions/limits/evidence, the 10-step dated timeline (outcome-led, chips, decision links), contributions with real status, hardware hypotheses, next steps.
  • Editorial text lives in site/content.json (operator-reviewed); charts and numbers in tables are computed from records.

Proposal (Claude)

local-ai-bench.org — information architecture proposal (draft by Claude)

Context

Static site generated from ~250 result records (tools/site/build.py). Audience, in order: (1) 0xSero, maintainer of the local-ai registry/recipe-kit and our lead, (2) potential fundraiser backers (hardware goals small/medium/big), (3) local-LLM practitioners who want a recipe for 3090-class boxes. The operator says the current pages "look very messy between them".

Current tabs (10 + 247 run pages)

Overview (per-model status cards, PR list, recent runs, a new headline callout) · Flash-Next leaderboard · GLM leaderboard · 27B leaderboard · Fresh-issues benchmark (method + results mixed) · Flash-Next vs 27B (public SWE-bench-Lite comparison, memorised) · Charts (8 charts with no narrative) · Runs (247) · Hardware & methodology (hardware, rules, decisions, spec). Problems: the story is nowhere; results for one question are spread over 4 tabs; three near-identical leaderboards; charts detached from the claims they support; method and results mixed; the old compare page contradicts the new benchmark unless read with a caveat.

What was done (chronological, all dated AEST, each backed by records + a decision file)
  1. Oct 3–4 · Box baseline: 3x RTX 3090 on AM4 (GPU2 x8 13.5 GB/s, GPU0 behind chipset 6 GB/s), faulty CPU core found and quarantined, boost off after machine-check errors (decisions 0004, 0010). Card order matters: GPU2-first +43–51 % prefill (0006).
  2. Oct 4 · Flash-Next on trellis/SGLang (Sero's PR #144 launch): lab 55 tok/s, prefill 2,400, but agentic floor ~40 at x8 — 50 not reached in any tested x8 config (0011). x16 card move dropped by the operator (0009).
  3. Oct 4–5 · GLM-5.3-Flash on one 3090: every route < 15 tok/s (0012).
  4. Oct 5 · PR #144 image OOM found (2/4 crashes) and mitigated with chunked prefill 8192 (0013). Public site launched.
  5. Oct 5–6 · Flash-Next vs 27B on public SWE-bench-Lite: 30/30 vs 18/30 — but Flash-Next reproduces upstream fixes verbatim (memorised) (0014).
  6. Oct 8 · From the Sero thread (0015): PR #144 rerun comment + kit submission PR posted; Strata (C++ GGUF engine) screened: Q2_0 floor 123 tok/s, IQ3_S 87 tok/s on one x8 card; moetier on the box: GLM NVMe recipe 6.1 tok/s at 55 GiB → 11.3 at 90 GiB; probe shows NVMe-bound → CPU-bound; trellis Flash-Next is PCIe-bound.
  7. Oct 8–10 · Built a private fresh-issues benchmark (0016): 430 attempts → 50 tasks, issue+PR after 1 Sep 2026, commitment hash published. Strata IQ3_S 48/50 vs 27B 36/50 (exact McNemar p = 0.0018, predeclared).
Proposed structure (5 top-level sections)
  1. Home = the story. Hero: the current answer in one sentence + 3–4 achievement tiles (fastest agentic setup on one 3090; best coding quality on fresh tasks; GLM verdict; contributions upstream). Then the dated timeline above as a vertical sequence, each step with its one key number, a status chip (result / negative result / fix / tool) and a link to the finding. Ends with "What's next" (Qwen 4 readiness, multi-language slice, fundraiser hardware sims).
  2. Findings (one page per question, narrative + the charts that support it + links to runs): a. Flash-Next speed on one RTX 3090 (engines compared; why x8 PCIe caps trellis; Strata result; stability fix) b. Coding quality (fresh-issues results; the memorisation finding on public tasks; 27B vs Flash-Next) c. GLM-5.3 on one RTX 3090 (routes tried; moetier NVMe recipe; what limits it; what hardware would fix it) d. Hardware lessons (PCIe lanes and card order, faulty core, NVMe at x2, RAM caps)
  3. Methods: how we measure (lab gates, kit sweep, TTFT, stability floor, quality panel, probe) and the fresh-issues benchmark construction (moved out of the results page), hardware record, rules for statuses.
  4. Data: one filterable config table (model/engine/quant/cards filters, replaces the three leaderboards), all runs, run detail pages, decisions log.
  5. Upstream: PRs and submissions to Sero's repos with status, and recipes ready to copy (launch commands).

Charts tab removed: every chart moves into the finding it supports. The old "Flash-Next vs 27B" page becomes a section of Findings b, framed as the memorisation evidence.

Questions for the reviewer

Is this the right structure for these audiences? What would you cut, merge or reorder? Is a timeline the right home page, or should the home lead with findings? Anything that makes the site look less credible?

Round 1 (GPT-6 Astra)

The proposal fixes the fragmentation, but risks replacing ten messy tabs with a polished lab diary. Lead with findings and usable recipes; make the chronology secondary. A public visitor needs to know what works, under which conditions, and how strong the evidence is.

0xSero’s priorities deserve a prominent contribution summary, not control of the entire navigation. Practitioners need reproducible setups; backers need evidence that a specific purchase would resolve a measured constraint. Neither audience benefits from reading seven dated steps first.

I would use five sections, in this order:

  1. Results — question-led findings, with a comparison summary.
  2. Recipes — tested launch commands, requirements, expected performance, limitations.
  3. Method & Hardware — measurement definitions, benchmark construction, exact machine configurations.
  4. Runs & Data — filterable results, individual runs, downloadable records, decisions.
  5. Project & Funding — upstream contributions, dated progress log, hardware proposals.

Home remains accessible through the logo. “Upstream” is insider language and combines two different jobs: tracking contributions and getting software running. Recipes should not be buried there.

The home page should contain, in order:

  • A bounded answer: which tested configuration you currently recommend for which workload, on what hardware, as of what date.
  • Three finding cards: practical agent performance, fresh-task coding results, and GLM feasibility. Each needs a configuration, metric, limitation, and evidence link.
  • “Run a tested setup” linking directly to recipes.
  • “What additional hardware would test” showing small/medium/big funding options, the measured constraint each addresses, cost, and success criterion.
  • “Recent changes” with three entries and a link to the full timeline.

Cut the achievement tiles. “Fastest agentic setup on one 3090” is especially risky when the supporting numbers mix laboratory throughput, agentic floors, engines, and quantizations. “Best coding quality” overstates a two-configuration, 50-task comparison. Use “Highest measured [defined metric] among tested configurations” and “48/50 versus 36/50 on this task set.” Put contributions below results, not alongside them as equivalent scientific achievements.

Keep the question-led findings, but sharpen their titles:

  • Which setup runs coding agents fastest?
  • How often do the tested setups solve fresh issues?
  • Is GLM practical on this machine?
  • Which hardware constraints affect performance?

The largest credibility issue is comparability, not navigation. “One RTX 3090” obscures a three-card host with unusual lane allocation, CPU restrictions, RAM requirements, and NVMe constraints. Display active GPUs separately from host hardware. Every comparison should expose model, engine, quantization, workload, context length, and relevant resource settings.

The fresh benchmark needs a compact evidence box beside its headline—not only a distant Methods link. Explain how 430 attempts yielded 50 tasks, exclusions and their timing, whether selection involved the evaluated systems, paired outcomes, and what was predeclared. A commitment hash needs a timestamp and a clear account of what it commits to. Recent issue dates alone do not establish contamination freedom; the p-value does not establish general coding superiority.

Keep chart integration, consolidated data, negative results, and the decisions log. Preserve old URLs with conspicuous superseded-result notices. Describe verbatim fixes as evidence consistent with memorisation unless stronger provenance establishes it. Treat proposed hardware remedies as hypotheses until tested.

Round 2 (GPT-6 Astra, after the owner's constraint)

The owner’s constraint changes my recommendation: make chronology the home page’s main structure, with the current answer and achievement summary above it. I withdraw the blanket objection to tiles; bounded result summaries serve a useful purpose.

1. Home, top to bottom

  • Current answer. A short, dated statement distinguishing the fastest tested configuration from the configuration evaluated for task success. Explicitly separate Q2_0’s speed result from IQ3_S’s 48/50 result. Name the workload, active GPU count, and three-card host; link to the applicable recipes.
  • Three achievement summaries: agentic performance, fresh-task outcomes, and GLM feasibility. Each gets a metric, essential conditions, limitation, and evidence link. Keep them compact enough that the answer and summaries fit a typical desktop first screen; on mobile, prioritize the answer. Put the fresh-benchmark evidence box directly beside or beneath its summary, not behind a tooltip.
  • “How we reached this answer.” The dated, chronological sequence is the page’s main body: 6–8 outcome-led steps, each linking to its finding and underlying records. Show the initial constraint, subsequent result, and any change in interpretation. Include the public-benchmark result together with the memorisation caveat.
  • Contribution summary. Show the stability work, rerun report, and kit submission with accurate statuses and links.
  • “Run a tested setup.” Direct recipe links with requirements and expected results.
  • “What additional hardware would test.” Measured constraints, proposed purchases, costs, hypotheses, and success criteria.

2. Final top-level navigation

Home → Findings → Recipes → Method & Hardware → Runs & Data → Funding & Contributions

Home deserves an explicit label because the owner makes the sequence a primary destination. Combining funding and contributions keeps project support together without burying recipes.

3. Remaining critical objections

Your timeline revision works, but “one number per step” should be a preference, not a rule. Some outcomes require a denominator, before/after values, or paired conditions. Clarity matters more than visual uniformity.

The proposed tiles still need corrections:

  • “Strata Q2_0” identifies an engine and quantization, not the model. Include the model and define “floor”; ten minutes alone does not explain that metric.
  • “GLM-5.3” must preserve the exact tested variant, apparently GLM-5.3-Flash, and say “across tested routes on this host.”
  • “Predeclared” must identify what was declared and when. The p-value belongs with paired outcomes and selection disclosures.
  • “2 upstream contributions” must distinguish submitted, accepted, and merged work.

Most critically, never combine Q2_0 speed and IQ3_S task success into an implied single configuration’s achievement.

Runs & Data › Decisions

T41 — Strata (Flash-Next GGUF) screen on GPU2 (decision 0015 item 3)

tickets/T41-strata-screen.md

T41 — Strata (Flash-Next GGUF) screen on GPU2 (decision 0015 item 3)

Pinned Strata v0.1.40.3 (d5ea713, local sm86 build), ISTA-DASLab GSQ-RCO GGUF quants at ed59f920, one RTX 3090 (GPU2, x8). Screen each quant with lab,kit,ttft,stab (same suite as Flash-Next trellis and 27B). No KL panel: Strata's CUDA server has no logprobs → quality = "unavailable via API"; quality comes from the coding evals instead. Order: Q2_0 → IQ3_S → Coder IQ1_M (pruned 256/512 experts) → IQ2_XS if useful. Finalists (best speed with tool calls clean) then run needle (ctx permitting) and the fresh-issues eval. Strata serves one request at a time: kit C2–C4 cells measure queueing, not batching. 2–3 cards only with a reason (e.g. to lift the 131k context). Claim to check: "100–140 tok/s on a 3090" (author estimate). Smoke 2026-10-08: Q2_0 130.5 tok/s on a 532-token code answer.

Runs & Data › Decisions

T42 — moetier on omarchy-gpu (decision 0015 item 2)

tickets/T42-moetier.md

T42 — moetier on omarchy-gpu (decision 0015 item 2)

Sero's ask: run github.com/0xSero/moetier (f751040) here. Hardware record: moetier/hardware/omarchy-gpu-rtx3090x8-kc3000.json.

  • GLM-5.3 NVMe recipe (glm53-rtx3090-55g-nvx4 adapted; image sybil-solutions/glm53-flash-offload v4.1-nvme 99b8926e): 55 GiB (his budget, sim ~7 tok/s here: single x2 NVMe) and 90 GiB (sim ~15) RAM caps. Steps lab,kit,panel,ttft with probe record over the whole server lifetime; panel = his GLM exl3 reference panel.
  • Flash-Next trellis c8192 with probe: utilization behind the ~40 tok/s agentic floor (chokepoints need his instrumented build, not public → utilization only).
  • Sims for fundraiser goals: research/moetier-onboarding.md §3.

Store: ~/models/local-ai-exp/nvx-glm53-v4.1 (117.3 GB, built 88 s, verify 2.48 GB/s).

Results (2026-10-08, GPU2 x8, boost off; probe = tools/probe_summary.py over moetier probe samples)

ConfigLab C1Lab prefillKit decode C1/C2/C3/C4 · C1@32kKit prefill 8k–64kPanelBinding (probe)
GLM v4.1 NVMe, 55 GiB (his budget)6.13145.2 / 4.85 / 4.71 / — · — (2-h step cap)181–340top-1 1.0, KL 0NVMe: 3.05 GB/s mean, ≥ 80 % of 3.36 ceiling 88 % of the kit; PCIe rx 3.3 GB/s; CPU lane 61–75 %
GLM v4.1 NVMe, 90 GiB11.363010.95 / 10.99 / 10.9 / 10.53 · 10.48379–676in bandCPU lane / host threads: lane 80–83 %, main+ctl spinning ~90 %; NVMe 1.3 GB/s (not saturated); PCIe rx 6.4 GB/s
Flash-Next trellis c8192 (probe arm)55.3–56.12,107–2,22248.5 / 53.5 / 55.8 / 53.9 · 49.12,279–2,435(not run)PCIe x8: rx mean 7.8–10 GB/s, p95 14.3 (= ceiling), ≥ 80 % of ceiling 20–43 % of the time; CPU 4 %, NVMe 0; agentic floor 39.0

Sim vs measured (moetier sim on trace G002): 55 GiB 7.0 → 6.1 (C1 lab) / 5.2 (kit); 90 GiB 15.0 → 11.3 / 10.95. The sim's CPU-lane cost is EPYC-calibrated (22 threads, 8-ch DDR4); ours is 13 lane threads at 3.4 GHz on 2-ch DDR4, so the sim is optimistic once the NVMe tier stops binding. Chokepoint attribution beyond utilization needs the engine live counters (his instrumented build; not public).

Runs & Data › Decisions

T43 — Runs on the private fresh-2026-09 slice (decision 0016)

tickets/T43-fresh-slice-runs.md

T43 — Runs on the private fresh-2026-09 slice (decision 0016)

50-task test slice (issue + PR created ≥ 2026-09-01, Opus-curated, validated gold 3/3 / empty 0, ≤ 3 per repo), frozen with a published commitment hash. Configs (GPU2): Strata IQ3_S (finalist), Qwen3.8-27B ctx131k, Strata Q2_0, Flash-Next trellis c8192. Primary comparison predeclared in 0016: Strata IQ3_S vs 27B (exact McNemar); others exploratory. Labels on every score: contamination-reduced, Opus-curated, semi-private for Anthropic models.

Freeze (2026-10-09 11:01 AEST)

  • Commitment: sha256(manifest bytes || 32-byte salt) = 5371c3b8d1caedc134657a17b86cad94fc7fe13d81365e932d068f5e22a9c710 (manifest + salt private; revealed when the slice is retired).
  • Test 50 (24 easy / 20 medium / 6 hard; 47 repos; 15 from repos in public SWE data), dev 10 (never reported).
  • Funnel: 430 attempts → clarity rejects 327 (tests misaligned 161, fix given away 95, underspecified 71) → 103 to setup/validation → 62 validated (setup/validation rejects: base eval error 7, test patch disables tests 7, no F2P 5, setup timeout 3, base test timeout 2, flaky F2P 1, gold eval error 1, disallowed pytest arg 1, agent gave up 1). Stage-2 wall 6.3 h, images 46.9 GB.

Strata IQ3_S (2026-10-09, 10 h, 83.0 tok/s)

48/50 solved (96 %); 3,366 tool calls, 1 parse failure, 1 truncation, 6 repetition hits. Leak checks: every test image has a clean tree at the base commit (git status empty); F2P test functions found in 12 images are pre-existing tests the hidden test patch modifies (normal). Gold overlap (share of upstream added lines reproduced verbatim): median 0.20, 3/50 ≥ 0.90 (vs median 1.00 on SWE-bench-Lite compare30) → not memorized. Caveat: the slice skews easy (median gold fix 7 added lines; 6/50 hard) — ceiling effect possible; 27B decides whether it discriminates.

Primary comparison and exploratory (2026-10-10)

ConfigSolved (50)easy / medium / hardDecode tok/sTool callsParse failTruncationsRepetition hits
Strata IQ3_S4823/24 · 19/20 · 6/683.03,366116
Strata Q2_046—123.12,74803—
Qwen3.8-27B ctx131k3619/24 · 14/20 · 3/6118.62,68243967
Flash-Next trellis c8192running

Primary (predeclared): Strata IQ3_S vs 27B — discordant 13 vs 1, exact McNemar p = 0.0018. Survives Bonferroni for 3 comparisons. Exploratory: Q2_0 vs 27B 10 vs 0 (p = 0.002); IQ3_S vs Q2_0 3 vs 1 (p = 0.63, no difference shown). Repo clustering: 47 repos for 50 tasks, so task-level ≈ repo-level here. Caveat on 27B: 39 output truncations and 67 repetition hits (vs 1 / 6) — part of its gap is looping/runaway generation under these sampling settings (temperature 0.7, max_tokens 8192), which is a property of the configuration as run. Labels: contamination-reduced, Opus-curated, semi-private for Anthropic models.

Flash-Next trellis c8192 on the slice: stopped by the operator at 9/50 (2026-10-10 13:20 AEST) — same weights as Strata at ~43 tok/s, not expected to differ measurably at n = 50; recorded as not completed (no result record).

Runs & Data › Decisions

T44 — Resource profiles: RAM, CPU, VRAM per phase (operator request 2026-10-10)

tickets/T44-resource-profiles.md

T44 — Resource profiles: RAM, CPU, VRAM per phase (operator request 2026-10-10)

What each setup uses while it works, on GPU2 (x8): idle (loaded), prefill (3 fresh ~32k-token prompts, 1 output token), decode (3 prompts, up to 2,048 output tokens). tools/measure/resources.py: moetier probe at 0.1 s (VRAM, GPU util, PCIe, per-CPU busy, container memory) + container memory.stat (anon vs page cache) at each phase end. Configs: Strata Q2_0, Strata IQ3_S, Qwen3.8-27B ctx131k, Flash-Next trellis c8192.

Runs & Data › Decisions

T45 — Strata Coder (Qwen3.8-Flash-Next Coder, GSQ-RCO IQ1_M): full suite (operator request 2026-10-11)

tickets/T45-strata-coder.md

T45 — Strata Coder (Qwen3.8-Flash-Next Coder, GSQ-RCO IQ1_M): full suite (operator request 2026-10-11)

ISTA-DASLab's expert-pruned Flash-Next (256 of 512 experts kept, ~3.5 bpw, 58.4 GB), on Strata v0.1.40.3, GPU2 x8. Steps: lab, TTFT, 10-min agentic stability, resource profile (idle/prefill/decode), needle (98k), SWE-bench-Lite compare30 (memorisation check), fresh-2026-09 test slice (exploratory vs the predeclared primary). Not applicable: kit sweep (SGLang-native API), reference panel (no logprobs).

Runs & Data › Decisions

T46 — OrcaRouter Qwen3.8-Flash-Next-Uncensored IQ3_XXS on Strata (operator request 2026-10-11)

tickets/T46-orcarouter-uncensored.md

T46 — OrcaRouter Qwen3.8-Flash-Next-Uncensored IQ3_XXS on Strata (operator request 2026-10-11)

orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4 IQ3_XXS (85.2 GB, 2 shards), Strata v0.1.40.3 via the documented manual compatibility pack (docs/ORCA.md, --compat-bf16), original MTP draft head, GPU2 x8, 131k context like the other Strata configs. Steps: lab, TTFT, 10-min agentic stability, resource profile, our fresh-issues benchmark (fresh-2026-09 test slice, exploratory vs the predeclared primary), then needle (98k) and SWE-bench-Lite compare30 (memorisation check). Not applicable: kit sweep (SGLang API), reference panel (no logprobs).

Results (2026-10-11/12, GPU2 x8)

  • Speed: lab C1 73.6–76.4, prefill 2,105–2,122, TTFT 8k/32k 2,007/2,162, agentic floor 86.0 (run marked failed: 2 repetition hits; 0/2 tasks in 10 min), resource profile decode 92.8 tok/s, 23.7 GiB VRAM, 51.8 GiB RAM, 15.3 cores.
  • Our fresh-issues benchmark: 14/50 (28 %) vs Strata IQ3_S 48/50, Q2_0 46/50, 27B 36/50 — 14 parse failures, 72 output truncations, 14 repetition hits, 2 agent format errors over 3,084 tool calls.
  • SWE-bench-Lite compare30: 21/30 (vs 28–30 for the GSQ-RCO quants), 5 parse failures, 18 truncations.
  • Needle: passes to 104,552 tokens.

Reading: the uncensored fine-tune (or its compat-BF16 repack on Strata) loses agentic reliability — runaway/looping replies and malformed tool calls — not speed. Not separable here: we did not run this GGUF on another engine.

Runs & Data › Decisions

T47 — Registry proofs: Strata Flash-Next IQ3_S and Q2_0 on RTX 3090 (operator-approved plan 2026-10-11)

tickets/T47-registry-strata-3090.md

T47 — Registry proofs: Strata Flash-Next IQ3_S and Q2_0 on RTX 3090 (operator-approved plan 2026-10-11)

Host-kind recipes for sybil-solutions/local-ai-registry (precedent: #205), served from our container built from Strata's Dockerfile (pinned d5ea713). tools/registry/strata_proof.sh runs the registry's lab.py try --on endpoint in the registry clone (~/dev/local-ai-exp/work/registry-strata-3090, branch rtx-3090-strata-flash-next), sets needs.host_ram_gb to the measured peak and re-proves so launch_sha256 matches. PR body: prs/registry-strata-3090.md.

Result

  • IQ3_S: 6/6 gates, 75.0 tok/s, prefill 2,174, needs.host_ram_gb 57.0 (peak 56.88), recipe qwen3.8-flash-next.strata.128k.json, launch_sha256 215f1674…
  • Q2_0: 6/6 gates, 99.7 tok/s, prefill 2,664, needs.host_ram_gb 41.0 (peak 40.44), recipe qwen3.8-flash-next.strata-q2.128k.json, launch_sha256 27cc9992…

Runs & Data › Decisions

T48 — Qwen3.8-27B with the registry's 208k RTX 3090 recipe on our fresh-issues benchmark

tickets/T48-27b-registry-208k-fresh.md

T48 — Qwen3.8-27B with the registry's 208k RTX 3090 recipe on our fresh-issues benchmark

Fairer 27B entry: the exact registry recipe (image c8922bd7…, 212,992 ctx, MTP, vision) instead of our 131k launch. Steps: lab, 10-min agentic stability, fresh-2026-09 test slice (same harness/sampling as T43). Exploratory vs the predeclared primary (which stays as run in T43).

Result (2026-10-12)

Registry 208k recipe: lab C1 126.3–138.7, prefill 831–836, agentic floor 90.1 (stability pass, 0 repetition hits). Fresh-issues benchmark: 39/50 (27 truncations, 43 repetition hits, 1 parse failure; 112 tok/s). Paired: Strata IQ3_S 48 vs 27B-208k 39 → discordant 10 vs 1, exact McNemar p = 0.0117 (exploratory). 27B-208k vs our 131k launch (36/50): 5 vs 2, p = 0.45 (no difference shown).

Runs & Data

Decisions and tickets

Every non-trivial choice is written down before or right after it is made, most with an independent model review attached.

Qwen3.8-Flash-Next configs

Model card →

One row per config_id, aggregating all of its records. Decode/prefill are the best value from a passing record (from a failed one when none passed, marked † and never counted as meeting a target). ✓ = meets the target (decode ≥ 50 tok/s, prefill > 1000 tok/s). Chips link to each run; ! = fail, ✕ = crash, – = skipped.

ConfigLab decode tok/sLab prefill tok/sTTFT prefill tok/sKit C1 tok/sOccupied maxCtx configuredQualityStabilityTool callsEngineQuantCardsLayout · PCIeEligibilityRuns
strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb103 ✓2,664 ✓–––131k–––strataGGUF Q2_01current · x8noneload load lab lab lab lab
strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 ! 1 fail100 ✓2,664 ✓2,620–125k131k–screened 1 run · min win 123116 ✓strataGGUF Q2_01current · x8noneload load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu293.2 ✓2,657 ✓2,639–106k131k–––strataGGUF IQ1_M1current · x8noneload lab TTFT
strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu278.1 ✓2,178 ✓2,036–123k131k–screened 1 run · min win 87.377 ✕strataGGUF IQ3_S1current · x8noneload load load load lab lab TTFT stability quality eval quality eval resources
strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 ! 1 fail76.4 ✓2,122 ✓2,007–128k131k–failed 1 run · min win 86.050 ✕strataGGUF IQ3_XXS1current · x8noneload lab lab TTFT stability ! quality eval quality eval needle resources
strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb75.0 ✓2,174 ✓–––131k–––strataGGUF IQ3_S1current · x8noneload lab
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 ✕ 2 crash ! 1 fail57.4 ✓2,537 ✓2,02147.8167k205kin band top-1 0.991 · KL 0.00095858failed 3 runs · min win 39.777 ✕sglangEXL3 3.05 bpw1current · x8noneload load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 ! 1 fail57.1 ✓608 ✕987–106k131k–failed 1 run · min win 38.670 ✕llama.cppGGUF IQ4_XS3current · 3× x8noneload lab lab TTFT stability !
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ! 5 fail56.4 ✓2,240 ✓2,29448.2191k205kin band top-1 0.99167 · KL 0.0009967failed 4 runs · min win 38.8256 ✓sglangEXL3 3.05 bpw1current · x8noneload load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 ! 1 fail56.1 ✓2,222 ✓–48.5167k205k–failed 1 run · min win 39.053 ✕sglangEXL3 3.05 bpw1current · x8noneload lab lab kit sweep stability !
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 ! 1 fail55.6 ✓2,596 ✓2,01748.2167k205kin band top-1 0.98867 · KL 0.00096647failed 1 run · min win 37.967 ✕sglangEXL3 3.05 bpw1current · x8noneload lab lab kit sweep TTFT stability ! quality panel
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 ! 1 fail55.6 ✓2,580 ✓2,014–167k205k–failed 1 run · min win 40.157 ✕sglangEXL3 3.05 bpw1current · x8noneload lab lab TTFT stability !
fn-trellis-3.05-0xsero-main-x8-gpu2 ! 1 fail55.3 ✓2,228 ✓2,26346.9167k205kin band top-1 0.99234 · KL 0.0009667failed 1 run · min win 38.163 ✕sglangEXL3 3.05 bpw1current · x8noneload lab lab kit sweep TTFT stability ! quality panel
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 ! 1 fail53.7 ✓2,596 ✓2,022–167k205k–failed 1 run · min win 40.972 ✕sglangEXL3 3.05 bpw1current · x8noneload lab lab TTFT stability !
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 ! 1 fail49.7 ✕2,564 ✓1,96943.7167k205kin band top-1 0.98834 · KL 0.00099656failed 1 run · min win 37.471 ✕sglangEXL3 3.05 bpw1current · x8noneload lab lab kit sweep TTFT stability ! quality panel
llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu01244.3 ✕717 ✕1,067–106k131k–––llama.cppGGUF IQ4_XS3current · 3× x8noneload lab TTFT
llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu01235.0 ✕713 ✕980–106k131k–––llama.cppGGUF IQ4_XS3current · 3× x8noneload lab TTFT
fn-llamacpp-iq4xs-128k-1x-fit-gpu2 ! 1 fail26.5 ✕157 ✕269–106k131k–failed 1 run · min win 23.849 ✕llama.cppGGUF IQ4_XS1current · x8noneload lab lab TTFT stability !
fn-llamacpp-iq4xs-128k-1x-gpu2 ✕ 1 crash–––––131k–––llama.cppGGUF IQ4_XS1current · x8noneload ✕
llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 ! 1 fail–––––131k–failed 1 run0 ✕llama.cppGGUF IQ4_XS3current · 3× x8noneload stability !
sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2 ✕ 1 crash–––––205k–––sglangEXL3 2.05 bpw1current · x8noneload ✕
sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2 ✕ 1 crash–––––205k–––sglangEXL3 4.05 bpw1current · x8noneload ✕
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 ✕ 2 crash–––––205k–––sglangEXL3 3.05 bpw1current · x8noneload lab ✕ TTFT ✕

GLM-5.3-Flash configs

Model card →

One row per config_id, aggregating all of its records. Decode/prefill are the best value from a passing record (from a failed one when none passed, marked † and never counted as meeting a target). ✓ = meets the target (decode ≥ 15 tok/s). Chips link to each run; ! = fail, ✕ = crash, – = skipped.

ConfigLab decode tok/sLab prefill tok/sTTFT prefill tok/sKit C1 tok/sOccupied maxCtx configuredQualityStabilityTool callsEngineQuantCardsLayout · PCIeEligibilityRuns
ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 ! 3 fail14.0 †189 †283–93k131k–failed 1 run · min win 13.30 ✕ik_llama.cppGGUF IQ2_M3current · 3× x8noneload lab ! lab ! TTFT stability !
llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 ✕ 1 crash ! 2 fail12.3 †158 †166–93k131k–failed 1 run · min win 11.426 ✕llama.cppGGUF IQ2_M3current · 3× x8noneload lab ! lab ✕ TTFT stability !
llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 ! 1 fail12.2 †153 †159–93k131k–––llama.cppGGUF IQ2_M3current · 3× x8noneload lab ! TTFT
llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 ! 3 fail11.8 †219 †240–93k131k–failed 1 run · min win 11.024 ✕llama.cppGGUF IQ2_M3current · 3× x8noneload lab ! lab ! TTFT stability !
glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 ! 1 fail11.3 †630 †33610.993k131k–––exllamav3?1current · x8noneload lab ! kit sweep TTFT quality panel
llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 ! 1 fail11.2 †258 †298–93k131k–––llama.cppGGUF IQ2_M3current · 3× x8noneload lab ! TTFT
llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 ! 3 fail10.5 †132 †130–93k131k–failed 1 run · min win 9.8622 ✕llama.cppGGUF IQ3_XXS3current · 3× x8noneload lab ! lab ! TTFT stability !
ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 ✕ 1 crash ! 1 fail10.2 †166 †248–93k131k–failed 1 run0 ✕ik_llama.cppGGUF IQ2_M3current · 3× x8noneload lab ! TTFT stability ✕
ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 ! 3 fail10.0 †151 †203–93k131k–failed 1 run · min win 9.460 ✕ik_llama.cppGGUF IQ2_M1current · x8noneload lab ! lab ! TTFT stability !
glm-llamacpp-iq2m-128k-1x-gpu2 ! 3 fail9.60 †81 †86.3–93k131k–failed 1 run · min win 8.9124 ✕llama.cppGGUF IQ2_M1current · x8noneload lab ! lab ! TTFT stability !
llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 ! 3 fail9.60 †288 †321–93k131k–failed 1 run · min win 8.8527 ✕llama.cppGGUF IQ2_M1current · x8noneload lab ! lab ! TTFT stability !
llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 ! 3 fail9.50 †200 †212–93k131k–failed 1 run · min win 8.8530 ✕llama.cppGGUF IQ2_M1current · x8noneload lab ! lab ! TTFT stability !
llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 ! 1 fail9.50 †201 †213–93k131k–––llama.cppGGUF IQ2_M1current · x8noneload lab ! TTFT
llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 ! 3 fail9.30 †80 †84.4–93k131k–failed 1 run · min win 8.8019 ✕llama.cppGGUF Q2_K1current · x8noneload lab ! lab ! TTFT stability !
llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 ! 1 fail9.10 †370 †384–93k131k–––llama.cppGGUF IQ2_M1current · x8noneload lab ! TTFT
ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 ✕ 1 crash ! 1 fail8.60 †147 †199–93k131k–failed 1 run0 ✕ik_llama.cppGGUF IQ2_M1current · x8noneload lab ! TTFT stability ✕
llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 ! 3 fail8.40 †71 †75.9–93k131k–failed 1 run · min win 7.6525 ✕llama.cppGGUF IQ3_XXS1current · x8noneload lab ! lab ! TTFT stability !
llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 ! 3 fail8.30 †156 †149–93k131k–failed 1 run · min win 7.8125 ✕llama.cppGGUF IQ4_XS3current · 3× x8noneload lab ! lab ! TTFT stability !
ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 ✕ 1 crash ! 1 fail8.20 †147 †201–93k131k–failed 1 run0 ✕ik_llama.cppGGUF IQ2_M1current · x8noneload lab ! TTFT stability ✕
exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 ! 4 fail7.50 †739 †483–93k131k–failed 1 run · min win 3.3621 ✕exllamav3EXL3 3.05 bpw1current · x8noneload load lab ! lab ! TTFT ! TTFT stability !
glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 ! 2 fail6.10 †314 †193–93k131k–––exllamav3?1current · x8noneload lab ! kit sweep ! TTFT quality panel
ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012 ✕ 1 crash–––––131k–––ik_llama.cppGGUF IQ2_M3current · 3× x8noneload ✕
ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012 ✕ 1 crash–––––131k–––ik_llama.cppGGUF IQ2_M3current · 3× x8noneload ✕
ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012 ✕ 1 crash–––––131k–––ik_llama.cppGGUF IQ2_M3current · 3× x8noneload ✕

Qwen3.8-27B configs

Model card →

One row per config_id, aggregating all of its records. Decode/prefill are the best value from a passing record (from a failed one when none passed, marked † and never counted as meeting a target). ✓ = meets the target (decode ≥ 15 tok/s). Chips link to each run; ! = fail, ✕ = crash, – = skipped.

ConfigLab decode tok/sLab prefill tok/sTTFT prefill tok/sKit C1 tok/sOccupied maxCtx configuredQualityStabilityTool callsEngineQuantCardsLayout · PCIeEligibilityRuns
sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 ! 3 fail144 ✓740––106k205k–––sglang-exl3EXL3 3.00 bpw1current · x8noneload load load load load lab ! lab quality eval needle ! needle ! resources
sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2139 ✓836––173k213k–screened 1 run · min win 90.1134 ✓sglang-exl3EXL3 3.00 bpw1current · x8noneload lab lab stability quality eval
sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 ! 4 fail––1,03512178k205k–failed 1 run · min win 111137 ✓sglang-exl3EXL3 3.00 bpw1current · x8noneload lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Superseded. This page is kept for old links; each chart now sits in the finding it supports.

Charts

Each chart appears once records contain its measurements. Hover or tap a mark for its run; every chart has a table view.

Agentic floor by config

The slowest 60-second window of decode during a 10-minute agentic coding run: what a coding agent actually gets. Best completed 10-minute run per model, quant and engine (runs below the 50 tok/s target are recorded as failed but count here).

Qwen3.8-Flash-NextGLM-5.3-FlashQwen3.8-27B
050100150Flash-Next GGUF Q2_0 · stratastrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2: slowest 60 s window 123 tok/s, 116 tool calls12327B EXL3 3.00 bpw · sglang-exl3sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2: slowest 60 s window 111 tok/s, 137 tool calls111Flash-Next GGUF IQ3_S · stratastrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2: slowest 60 s window 87.3 tok/s, 77 tool calls87.3Flash-Next GGUF IQ3_XXS · stratastrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2: slowest 60 s window 86.0 tok/s, 50 tool calls86.0Flash-Next EXL3 3.05 bpw · trellis/SGLangsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2: slowest 60 s window 40.9 tok/s, 72 tool calls40.9Flash-Next GGUF IQ4_XS · llama.cpp · 3 cardsllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: slowest 60 s window 38.6 tok/s, 70 tool calls38.6Flash-Next GGUF IQ4_XS · llama.cppfn-llamacpp-iq4xs-128k-1x-fit-gpu2: slowest 60 s window 23.8 tok/s, 49 tool calls23.8GLM GGUF IQ2_M · ik_llama.cpp · 3 cardsik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: slowest 60 s window 13.3 tok/s, 0 tool calls13.3GLM GGUF IQ2_M · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012: slowest 60 s window 11.4 tok/s, 26 tool calls11.4GLM GGUF IQ3_XXS · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012: slowest 60 s window 9.86 tok/s, 22 tool calls9.86GLM GGUF IQ2_M · ik_llama.cppik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2: slowest 60 s window 9.46 tok/s, 0 tool calls9.46GLM GGUF IQ2_M · llama.cppglm-llamacpp-iq2m-128k-1x-gpu2: slowest 60 s window 8.91 tok/s, 24 tool calls8.91GLM GGUF Q2_K · llama.cppllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2: slowest 60 s window 8.80 tok/s, 19 tool calls8.80GLM GGUF IQ4_XS · llama.cpp · 3 cardsllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012: slowest 60 s window 7.81 tok/s, 25 tool calls7.81target 50tok/s, slowest 60-second window of a 10-minute agentic coding run
Table view
ConfigFloor tok/sRun
Flash-Next GGUF Q2_0 · strata123strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
27B EXL3 3.00 bpw · sglang-exl3111sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Flash-Next GGUF IQ3_S · strata87.3strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Flash-Next GGUF IQ3_XXS · strata86.0strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Flash-Next EXL3 3.05 bpw · trellis/SGLang40.9sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Flash-Next GGUF IQ4_XS · llama.cpp · 3 cards38.6llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Flash-Next GGUF IQ4_XS · llama.cpp23.8fn-llamacpp-iq4xs-128k-1x-fit-gpu2
GLM GGUF IQ2_M · ik_llama.cpp · 3 cards13.3ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
GLM GGUF IQ2_M · llama.cpp · 3 cards11.4llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
GLM GGUF IQ3_XXS · llama.cpp · 3 cards9.86llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
GLM GGUF IQ2_M · ik_llama.cpp9.46ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
GLM GGUF IQ2_M · llama.cpp8.91glm-llamacpp-iq2m-128k-1x-gpu2
GLM GGUF Q2_K · llama.cpp8.80llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
GLM GGUF IQ4_XS · llama.cpp · 3 cards7.81llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
GLM GGUF IQ3_XXS · llama.cpp7.65llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
GLM EXL3 3.05 bpw · exllamav33.36exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

What limits each setup

Host-side utilization sampled every 0.1 s during the run (moetier probe), as a share of this box's measured ceilings. The bar nearest 100 % is the bottleneck.

GPU busyPCIe host→GPU, % of measured 13.5 GB/sNVMe read, % of 3.36 GB/sCPU busy
0%20%40%60%80%100%GLM-5.3 · NVMe recipe · 55 GiB RAM: GPU busy 87 % of samples with a kernel running87%GLM-5.3 · NVMe recipe · 55 GiB RAM: PCIe in 3.34 GB/s of 13.5 (p95 6.17)25%GLM-5.3 · NVMe recipe · 55 GiB RAM: NVMe read 3.08 GB/s of 3.3692%GLM-5.3 · NVMe recipe · 55 GiB RAM: CPU 62% busy (expert-compute cores)62%GLM-5.3 55 GiBGLM-5.3 · NVMe recipe · 90 GiB RAM: GPU busy 96 % of samples with a kernel running96%GLM-5.3 · NVMe recipe · 90 GiB RAM: PCIe in 6.42 GB/s of 13.5 (p95 14.27)48%GLM-5.3 · NVMe recipe · 90 GiB RAM: NVMe read 1.31 GB/s of 3.3639%GLM-5.3 · NVMe recipe · 90 GiB RAM: CPU 83% busy (expert-compute cores)83%GLM-5.3 90 GiBFlash-Next · trellis · agentic run: GPU busy 89 % of samples with a kernel running89%Flash-Next · trellis · agentic run: PCIe in 7.77 GB/s of 13.5 (p95 14.25)58%Flash-Next · trellis · agentic run: NVMe read 0.0 GB/s of 3.360%Flash-Next · trellis · agentic run: CPU 4% busy (all cores)4%Flash-Next agenticshare of ceiling
Table view
ArmPhaseGPUPCIeNVMeCPU
GLM-5.3 · NVMe recipe · 55 GiB RAMkit87 % of samples with a kernel running3.34 GB/s of 13.5 (p95 6.17)3.08 GB/s of 3.3662% busy (expert-compute cores)
GLM-5.3 · NVMe recipe · 90 GiB RAMkit96 % of samples with a kernel running6.42 GB/s of 13.5 (p95 14.27)1.31 GB/s of 3.3683% busy (expert-compute cores)
Flash-Next · trellis · agentic runstab89 % of samples with a kernel running7.77 GB/s of 13.5 (p95 14.25)0.0 GB/s of 3.364% busy (all cores)

Decode vs occupied context

Lab C1 decode and the stability minimum 60 s window against the occupied prompt tokens reported by the server.

Qwen3.8-Flash-NextGLM-5.3-FlashQwen3.8-27Blab C1 (first 30 s)stability: min 60 s window
050100150200050100150200occupied context (k tokens)decode tok/sFlash-Next target 50GLM gate 15128kfn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 49.3 tok/s at 166,667 occupied · passfn-trellis-3.05-0xsero-main-x8-gpu2 · stability min 60 s window 38.1 tok/s at 30,034 occupied · failfn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · passfn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.5 tok/s at 106,294 occupied · passfn-llamacpp-iq4xs-128k-1x-fit-gpu2 · stability min 60 s window 23.8 tok/s at 16,819 occupied · failfn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.1 tok/s at 106,294 occupied · passglm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failglm-llamacpp-iq2m-128k-1x-gpu2 · stability min 60 s window 8.91 tok/s at 9,047 occupied · failglm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.20 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · stability min 60 s window 8.80 tok/s at 6,658 occupied · failllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.30 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 10.0 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · stability min 60 s window 9.46 tok/s at 1,874 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 · lab C1 decode 8.20 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.30 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · stability min 60 s window 7.65 tok/s at 4,875 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.40 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · stability min 60 s window 8.85 tok/s at 9,732 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 · lab C1 decode 8.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · lab C1 decode 12.3 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · stability min 60 s window 11.4 tok/s at 8,118 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.5 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · stability min 60 s window 9.86 tok/s at 5,309 occupied · failllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.4 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 · lab C1 decode 12.2 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 11.0 tok/s at 6,174 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · stability min 60 s window 8.85 tok/s at 6,632 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.00 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 7.81 tok/s at 6,852 occupied · failllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.30 tok/s at 93,437 occupied · failllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 56.9 tok/s at 106,294 occupied · passllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · stability min 60 s window 38.6 tok/s at 39,226 occupied · failllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 57.1 tok/s at 106,294 occupied · passllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 · lab C1 decode 9.10 tok/s at 93,437 occupied · failllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 · lab C1 decode 11.2 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 14.0 tok/s at 93,437 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · stability min 60 s window 13.3 tok/s at 1,874 occupied · failik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 13.4 tok/s at 93,437 occupied · failllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 · lab C1 decode 44.3 tok/s at 106,294 occupied · passllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 · lab C1 decode 35.0 tok/s at 106,294 occupied · passik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 · lab C1 decode 10.2 tok/s at 93,437 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · stability min 60 s window 37.9 tok/s at 36,327 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 54.5 tok/s at 166,667 occupied · passexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.40 tok/s at 93,437 occupied · failexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · stability min 60 s window 3.36 tok/s at 4,589 occupied · failexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.50 tok/s at 93,437 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 55.8 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · stability min 60 s window 39.7 tok/s at 25,309 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 57.4 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · stability min 60 s window 40.1 tok/s at 29,027 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 53.7 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · stability min 60 s window 40.9 tok/s at 39,281 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 51.3 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 48.1 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · stability min 60 s window 37.4 tok/s at 34,417 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 49.7 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.9 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.5 tok/s at 25,179 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 49.5 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 49.9 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 53.5 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 38.8 tok/s at 28,209 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 56.4 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 55.0 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 40.4 tok/s at 30,539 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 45.0 tok/s at 166,667 occupied · passsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 · stability min 60 s window 111 tok/s at 52,045 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 48.8 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.8 tok/s at 28,542 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · passsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 · lab C1 decode 144 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 123 tok/s at 38,712 occupied · passstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · passglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 · lab C1 decode 6.10 tok/s at 93,437 occupied · failglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 · lab C1 decode 11.3 tok/s at 93,437 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · passsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · stability min 60 s window 39.0 tok/s at 28,256 occupied · failsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 56.1 tok/s at 166,667 occupied · passstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 78.1 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 87.3 tok/s at 39,624 occupied · passstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 75.5 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 · lab C1 decode 93.2 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 76.4 tok/s at 106,294 occupied · passstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 86.0 tok/s at 56,311 occupied · failstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 73.6 tok/s at 106,294 occupied · passsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 126 tok/s at 173,372 occupied · passsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · stability min 60 s window 90.1 tok/s at 68,013 occupied · passsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 139 tok/s at 173,372 occupied · pass
Table view
ModelConfigMeasureOccupiedtok/sOutcome
Qwen3.8-Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2lab C1 decode166,66749.3✓ pass
Qwen3.8-Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2stability min 60 s window30,03438.1! fail
Qwen3.8-Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2lab C1 decode166,66755.3✓ pass
Qwen3.8-Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2lab C1 decode106,29426.5✓ pass
Qwen3.8-Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2stability min 60 s window16,81923.8! fail
Qwen3.8-Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2lab C1 decode106,29426.1✓ pass
GLM-5.3-Flashglm-llamacpp-iq2m-128k-1x-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashglm-llamacpp-iq2m-128k-1x-gpu2stability min 60 s window9,0478.91! fail
GLM-5.3-Flashglm-llamacpp-iq2m-128k-1x-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab C1 decode93,4379.20! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2stability min 60 s window6,6588.80! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab C1 decode93,4379.30! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab C1 decode93,43710.0! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2stability min 60 s window1,8749.46! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2lab C1 decode93,4378.20! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab C1 decode93,4378.30! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2stability min 60 s window4,8757.65! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab C1 decode93,4378.40! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2stability min 60 s window9,7328.85! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2lab C1 decode93,4378.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012lab C1 decode93,43712.3! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012stability min 60 s window8,11811.4! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab C1 decode93,43710.5! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012stability min 60 s window5,3099.86! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab C1 decode93,43710.4! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012lab C1 decode93,43712.2! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,43711.8! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012stability min 60 s window6,17411.0! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,43711.8! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2lab C1 decode93,4379.50! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2stability min 60 s window6,6328.85! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab C1 decode93,4379.60! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,4378.00! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012stability min 60 s window6,8527.81! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab C1 decode93,4378.30! fail
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012lab C1 decode106,29456.9✓ pass
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012stability min 60 s window39,22638.6! fail
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012lab C1 decode106,29457.1✓ pass
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2lab C1 decode93,4379.10! fail
GLM-5.3-Flashllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012lab C1 decode93,43711.2! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab C1 decode93,43714.0! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012stability min 60 s window1,87413.3! fail
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab C1 decode93,43713.4! fail
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012lab C1 decode106,29444.3✓ pass
Qwen3.8-Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012lab C1 decode106,29435.0✓ pass
GLM-5.3-Flashik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012lab C1 decode93,43710.2! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2lab C1 decode166,66755.6✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2stability min 60 s window36,32737.9! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2lab C1 decode166,66754.5✓ pass
GLM-5.3-Flashexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab C1 decode93,4377.40! fail
GLM-5.3-Flashexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2stability min 60 s window4,5893.36! fail
GLM-5.3-Flashexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab C1 decode93,4377.50! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab C1 decode166,66755.8✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2stability min 60 s window25,30939.7! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab C1 decode166,66757.4✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2lab C1 decode166,66755.6✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2stability min 60 s window29,02740.1! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2lab C1 decode166,66754.6✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2lab C1 decode166,66753.7✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2stability min 60 s window39,28140.9! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2lab C1 decode166,66751.3✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2lab C1 decode166,66748.1✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2stability min 60 s window34,41737.4! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2lab C1 decode166,66749.7✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66754.9✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window25,17939.5! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66749.5✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab C1 decode166,66749.9✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66753.5✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window28,20938.8! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66756.4✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66755.0✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window30,53940.4! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66745.0✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2stability min 60 s window52,045111! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66748.8✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2stability min 60 s window28,54239.8! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab C1 decode166,66754.6✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2lab C1 decode106,294144✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2lab C1 decode106,294100✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2stability min 60 s window38,712123✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2lab C1 decode106,294100✓ pass
GLM-5.3-Flashglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2lab C1 decode93,4376.10! fail
GLM-5.3-Flashglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2lab C1 decode93,43711.3! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2lab C1 decode166,66755.3✓ pass
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2stability min 60 s window28,25639.0! fail
Qwen3.8-Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2lab C1 decode166,66756.1✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2lab C1 decode106,29478.1✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2stability min 60 s window39,62487.3✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2lab C1 decode106,29475.5✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2lab C1 decode106,29493.2✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2lab C1 decode106,29476.4✓ pass
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2stability min 60 s window56,31186.0! fail
Qwen3.8-Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2lab C1 decode106,29473.6✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab C1 decode173,372126✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2stability min 60 s window68,01390.1✓ pass
Qwen3.8-27Bsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab C1 decode173,372139✓ pass

Speed vs quality

Best lab decode per config against its quality measurement.

Qwen3.8-Flash-Next: decode vs mean KL (lower KL is better)

02040608000.00050.0010.0015mean KL vs exllamav3 reference panellab decode tok/starget 50band KL ≤ 0.0013fn-trellis-3.05-0xsero-main-x8-gpu2 · KL 0.0009667 · top-1 0.99234 · decode 55.3fn-trellis-3.05-0xsero-main-x8-gpu2sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · KL 0.00096647 · top-1 0.98867 · decode 55.6sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · KL 0.00095858 · top-1 0.991 · decode 57.4sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · KL 0.00099656 · top-1 0.98834 · decode 49.7sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · KL 0.0009967 · top-1 0.99167 · decode 56.4sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Table view
ConfigKLtop-1Lab decode
fn-trellis-3.05-0xsero-main-x8-gpu20.00096670.9923455.3
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu20.000966470.9886755.6
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu20.000958580.99157.4
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu20.000996560.9883449.7
sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu20.00099670.9916756.4

1 vs 3 cards (current layout)

Best config per card count, same model.

Best lab decode

050100150Flash-Next GGUF Q2_0 · 1 cardstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb: 103 tok/s decode103Flash-Next GGUF IQ4_XS · 3 cardsllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: 57.1 tok/s decode57.1GLM ? · 1 cardglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2: 11.3 tok/s decode11.3GLM GGUF IQ2_M · 3 cardsik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: 14.0 tok/s decode14.0tok/s

Lab prefill of the same configs

01k2k3kFlash-Next GGUF Q2_0 · 1 cardstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb: 2,664 tok/s prefill2,664Flash-Next GGUF IQ4_XS · 3 cardsllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: 608 tok/s prefill608GLM ? · 1 cardglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2: 630 tok/s prefill630GLM GGUF IQ2_M · 3 cardsik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: 189 tok/s prefill189tok/s
Table view

x8 vs x16

No measurements yet: needs single-card lab results in both layouts for a model.

All runs

Every record, newest first, failed, crashed and skipped runs included.

StartedOutcomeModelKindConfigKey metricsStageTicket
2026-10-12 11:48 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 99.7 tok/s · lab_prefill_tps 2,664 tok/sT47
2026-10-12 11:48 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gbload_seconds 25.0 sT47
2026-10-12 05:49 AEST✓ pass27Bquality evalsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2solve_rate 0.78 fractionT48
2026-10-12 05:44 AEST✓ pass27Blabsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab_decode_c1_tps 139 tok/s · lab_prefill_tps 836 tok/sT48
2026-10-12 05:33 AEST✓ pass27Bstabilitysglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2decode_window_min_tps 90.1 tok/s · tool_calls 134 callsT48
2026-10-12 05:28 AEST✓ pass27Blabsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2lab_decode_c1_tps 126 tok/s · lab_prefill_tps 831 tok/sT48
2026-10-12 05:26 AEST✓ pass27Bloadsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2load_seconds 113 sT48
2026-10-12 05:23 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 88.2 tok/s · lab_prefill_tps 2,638 tok/sT47
2026-10-12 05:20 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 101 tok/s · lab_prefill_tps 2,631 tok/sT47
2026-10-12 05:18 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gblab_decode_c1_tps 103 tok/s · lab_prefill_tps 2,644 tok/sT47
2026-10-12 05:17 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gbload_seconds 25.0 sT47
2026-10-12 05:15 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gblab_decode_c1_tps 75.0 tok/s · lab_prefill_tps 2,174 tok/sT47
2026-10-12 05:14 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gbload_seconds 34.1 sT47
2026-10-12 00:44 AEST✓ passFlash-Nextquality evalstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2solve_rate 0.7 fractionT46
2026-10-12 00:41 AEST✓ passFlash-Nextneedlestrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2T46
2026-10-11 12:13 AEST✓ passFlash-Nextquality evalstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2solve_rate 0.28 fractionT46
2026-10-11 12:11 AEST✓ passFlash-Nextresourcesstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2T46
2026-10-11 12:06 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 73.6 tok/s · lab_prefill_tps 2,105 tok/sT46
2026-10-11 11:56 AEST! failFlash-Nextstabilitystrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2decode_window_min_tps 86.0 tok/s · tool_calls 50 callsT46
2026-10-11 11:55 AEST✓ passFlash-NextTTFTstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2ttft_prefill_tps 2,007 tok/sT46
2026-10-11 11:48 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 76.4 tok/s · lab_prefill_tps 2,122 tok/sT46
2026-10-11 11:47 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2load_seconds 34.1 sT46
2026-10-11 10:50 AEST✓ passFlash-NextTTFTstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2ttft_prefill_tps 2,639 tok/sT45
2026-10-11 10:48 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 93.2 tok/s · lab_prefill_tps 2,657 tok/sT45
2026-10-11 10:47 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2load_seconds 28.1 sT45
2026-10-10 21:00 AEST✓ passFlash-Nextresourcessglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2T44
2026-10-10 20:56 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 264 sT44
2026-10-10 20:53 AEST✓ pass27Bresourcessglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2T44
2026-10-10 20:51 AEST✓ pass27Bloadsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2load_seconds 131 sT44
2026-10-10 20:48 AEST✓ passFlash-Nextresourcesstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2T44
2026-10-10 20:48 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2load_seconds 31.1 sT44
2026-10-10 20:46 AEST✓ passFlash-Nextresourcesstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2T44
2026-10-10 20:46 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2load_seconds 25.1 sT44
2026-10-10 10:23 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 267 sT43
2026-10-10 03:48 AEST✓ passFlash-Nextquality evalstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2solve_rate 0.92 fractionT43
2026-10-10 03:48 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2load_seconds 24.9 sT43
2026-10-09 21:38 AEST✓ pass27Bquality evalsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2solve_rate 0.72 fractionT43
2026-10-09 21:36 AEST✓ pass27Bloadsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2load_seconds 131 sT43
2026-10-09 11:02 AEST✓ passFlash-Nextquality evalstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2solve_rate 0.96 fractionT43
2026-10-09 11:01 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2load_seconds 31.0 sT43
2026-10-09 02:28 AEST✓ passFlash-Nextquality evalstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2solve_rate 0.9667 fractionT41
2026-10-09 02:28 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2load_seconds 30.9 sT41
2026-10-08 23:43 AEST✓ passFlash-Nextquality evalstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2solve_rate 0.9333 fractionT41
2026-10-08 23:43 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2load_seconds 24.9 sT41
2026-10-08 23:39 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 75.5 tok/s · lab_prefill_tps 2,178 tok/sT41
2026-10-08 23:28 AEST✓ passFlash-Nextstabilitystrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2decode_window_min_tps 87.3 tok/s · tool_calls 77 callsT41
2026-10-08 23:27 AEST✓ passFlash-NextTTFTstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2ttft_prefill_tps 2,036 tok/sT41
2026-10-08 23:23 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 78.1 tok/s · lab_prefill_tps 2,174 tok/sT41
2026-10-08 23:22 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2load_seconds 43.0 sT41
2026-10-08 23:05 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2lab_decode_c1_tps 56.1 tok/s · lab_prefill_tps 2,222 tok/sT42
2026-10-08 22:55 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2decode_window_min_tps 39.0 tok/s · tool_calls 53 callsT42
2026-10-08 22:40 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2decode_c1_tps 48.5 tok/s · prefill_32k_tps 2,435 tok/sT42
2026-10-08 22:36 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2lab_decode_c1_tps 55.3 tok/s · lab_prefill_tps 2,107 tok/sT42
2026-10-08 22:31 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2load_seconds 260 sT42
2026-10-08 22:26 AEST✓ passGLMTTFTglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2ttft_prefill_tps 336 tok/sT42
2026-10-08 22:23 AEST✓ passGLMquality panelglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2top1_agreement 1.00 fraction · mean_kl 0 natsT42
2026-10-08 20:52 AEST✓ passGLMkit sweepglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2decode_c1_tps 10.9 tok/s · prefill_32k_tps 621 tok/sT42
2026-10-08 20:38 AEST! failGLMlabglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2lab_decode_c1_tps 11.3 tok/s · lab_prefill_tps 630 tok/sgatesT42
2026-10-08 20:36 AEST✓ passGLMloadglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2load_seconds 113 sT42
2026-10-08 20:25 AEST✓ passGLMTTFTglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2ttft_prefill_tps 193 tok/sT42
2026-10-08 19:34 AEST✓ passGLMquality panelglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2top1_agreement 1.00 fraction · mean_kl 0 natsT42
2026-10-08 17:34 AEST! failGLMkit sweepglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2decode_c1_tps unavailable · prefill_32k_tps unavailabletimeoutT42
2026-10-08 16:34 AEST! failGLMlabglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2lab_decode_c1_tps 6.10 tok/s · lab_prefill_tps 314 tok/sgatesT42
2026-10-08 16:32 AEST✓ passGLMloadglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2load_seconds 101 sT42
2026-10-08 16:29 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 100 tok/s · lab_prefill_tps 2,657 tok/sT41
2026-10-08 16:18 AEST✓ passFlash-Nextstabilitystrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2decode_window_min_tps 123 tok/s · tool_calls 116 callsT41
2026-10-08 16:17 AEST✓ passFlash-NextTTFTstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2ttft_prefill_tps 2,620 tok/sT41
2026-10-08 16:17 AEST! failFlash-Nextkit sweepstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2decode_c1_tps unavailable · prefill_32k_tps unavailablemeasureT41
2026-10-08 16:14 AEST✓ passFlash-Nextlabstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2lab_decode_c1_tps 100 tok/s · lab_prefill_tps 2,664 tok/sT41
2026-10-08 16:14 AEST✓ passFlash-Nextloadstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2load_seconds 24.9 sT41
2026-10-06 03:41 AEST! fail27Bneedlesglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2measureT40
2026-10-06 03:37 AEST✓ pass27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2lab_decode_c1_tps 144 tok/s · lab_prefill_tps 740 tok/sT40
2026-10-06 03:35 AEST✓ pass27Bloadsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2load_seconds 128 sT40
2026-10-06 03:30 AEST✓ passFlash-Nextneedlesglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2T40
2026-10-06 03:25 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 260 sT40
2026-10-06 03:18 AEST! fail27Bneedlesglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2measureT40
2026-10-06 03:16 AEST✓ pass27Bloadsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2load_seconds 128 sT40
2026-10-06 03:15 AEST! fail27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailablegatesT40
2026-10-06 03:13 AEST✓ pass27Bloadsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2load_seconds 128 sT40
2026-10-05 21:50 AEST✓ passFlash-Nextquality evalsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2solve_rate 1.00 fractionT40
2026-10-05 21:45 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 54.6 tok/s · lab_prefill_tps 2,234 tok/sT40
2026-10-05 21:35 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 39.8 tok/s · tool_calls 69 callsT40
2026-10-05 21:32 AEST! failFlash-Nextneedlesglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2measureT40
2026-10-05 21:31 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2ttft_prefill_tps 2,284 tok/sT40
2026-10-05 21:16 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_c1_tps 47.5 tok/s · prefill_32k_tps 2,444 tok/sT40
2026-10-05 21:10 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 48.8 tok/s · lab_prefill_tps 2,131 tok/sT40
2026-10-05 21:06 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 263 sT40
2026-10-05 18:36 AEST✓ pass27Bquality evalsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2solve_rate 0.6 fractionT40
2026-10-05 18:35 AEST! fail27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailablegatesT40
2026-10-05 18:25 AEST! fail27Bstabilitysglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2decode_window_min_tps 111 tok/s · tool_calls 137 callsT40
2026-10-05 18:25 AEST! fail27Bneedlesglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2measureT40
2026-10-05 18:21 AEST✓ pass27BTTFTsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2ttft_prefill_tps 1,035 tok/sT40
2026-10-05 18:08 AEST✓ pass27Bkit sweepsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2decode_c1_tps 121 tok/s · prefill_32k_tps 970 tok/sT40
2026-10-05 18:07 AEST! fail27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailablegatesT40
2026-10-05 18:05 AEST✓ pass27Bloadsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2load_seconds 128 sT40
2026-10-05 12:24 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 45.0 tok/s · lab_prefill_tps 2,240 tok/sT14a
2026-10-05 12:13 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 40.4 tok/s · tool_calls 61 callsT14a
2026-10-05 11:58 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_c1_tps 48.1 tok/s · prefill_32k_tps 2,434 tok/sT14a
2026-10-05 11:58 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2top1_agreement 0.99167 fraction · mean_kl 0.0009967 natsT14a
2026-10-05 11:56 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2ttft_prefill_tps 2,289 tok/sT14a
2026-10-05 11:50 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 55.0 tok/s · lab_prefill_tps 2,131 tok/sT14a
2026-10-05 11:45 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 260 sT14a
2026-10-05 11:38 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 56.4 tok/s · lab_prefill_tps 2,228 tok/sT14a
2026-10-05 11:27 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 38.8 tok/s · tool_calls 62 callsT14a
2026-10-05 11:27 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2top1_agreement 0.98934 fraction · mean_kl 0.00090651 natsT14a
2026-10-05 11:25 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2ttft_prefill_tps 2,283 tok/sT14a
2026-10-05 11:10 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_c1_tps 47.0 tok/s · prefill_32k_tps 2,440 tok/sT14a
2026-10-05 11:03 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 53.5 tok/s · lab_prefill_tps 2,134 tok/sT14a
2026-10-05 10:59 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 257 sT14a
2026-10-05 10:49 AEST✕ crashFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_window_min_tps unavailable · tool_calls 6 callsmeasureT13
2026-10-05 10:48 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2top1_agreement 0.991 fraction · mean_kl 0.00095858 natsT13
2026-10-05 10:47 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2ttft_prefill_tps 2,021 tok/sT13
2026-10-05 10:32 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_c1_tps 47.8 tok/s · prefill_32k_tps 2,669 tok/sT13
2026-10-05 10:25 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab_decode_c1_tps 49.9 tok/s · lab_prefill_tps 2,419 tok/sT13
2026-10-05 10:21 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2load_seconds 263 sT13
2026-10-05 10:17 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 49.5 tok/s · lab_prefill_tps 2,237 tok/sT14a
2026-10-05 10:07 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 39.5 tok/s · tool_calls 64 callsT14a
2026-10-05 10:06 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2top1_agreement 0.98934 fraction · mean_kl 0.00091506 natsT14a
2026-10-05 10:05 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2ttft_prefill_tps 2,294 tok/sT14a
2026-10-05 09:49 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_c1_tps 48.2 tok/s · prefill_32k_tps 2,436 tok/sT14a
2026-10-05 09:41 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2lab_decode_c1_tps 54.9 tok/s · lab_prefill_tps 2,131 tok/sT14a
2026-10-05 09:37 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2load_seconds 260 sT14a
2026-10-05 09:33 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2lab_decode_c1_tps 49.7 tok/s · lab_prefill_tps 2,564 tok/sT14a
2026-10-05 09:22 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2decode_window_min_tps 37.4 tok/s · tool_calls 71 callsT14a
2026-10-05 09:21 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2top1_agreement 0.98834 fraction · mean_kl 0.00099656 natsT14a
2026-10-05 09:20 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2ttft_prefill_tps 1,969 tok/sT14a
2026-10-05 09:04 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2decode_c1_tps 43.7 tok/s · prefill_32k_tps 2,644 tok/sT14a
2026-10-05 09:00 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2lab_decode_c1_tps 48.1 tok/s · lab_prefill_tps 2,408 tok/sT14a
2026-10-05 08:55 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2load_seconds 260 sT14a
2026-10-05 00:18 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab_decode_c1_tps 55.8 tok/s · lab_prefill_tps 2,419 tok/sT13
2026-10-05 00:14 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2load_seconds 257 sT13
2026-10-05 00:07 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2lab_decode_c1_tps 51.3 tok/s · lab_prefill_tps 2,596 tok/sT14b
2026-10-04 23:57 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2decode_window_min_tps 40.9 tok/s · tool_calls 72 callsT14b
2026-10-04 23:55 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2ttft_prefill_tps 2,022 tok/sT14b
2026-10-04 23:52 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2lab_decode_c1_tps 53.7 tok/s · lab_prefill_tps 2,440 tok/sT14b
2026-10-04 23:48 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2load_seconds 215 sT14b
2026-10-04 23:43 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2lab_decode_c1_tps 54.6 tok/s · lab_prefill_tps 2,580 tok/sT14a
2026-10-04 23:33 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2decode_window_min_tps 40.1 tok/s · tool_calls 57 callsT14a
2026-10-04 23:31 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2ttft_prefill_tps 2,014 tok/sT14a
2026-10-04 23:26 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2lab_decode_c1_tps 55.6 tok/s · lab_prefill_tps 2,430 tok/sT14a
2026-10-04 23:22 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2load_seconds 254 sT14a
2026-10-04 23:13 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab_decode_c1_tps 57.4 tok/s · lab_prefill_tps 2,537 tok/sT14a
2026-10-04 23:02 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_window_min_tps 39.7 tok/s · tool_calls 71 callsT14a
2026-10-04 22:58 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2load_seconds 264 sT14a
2026-10-04 22:54 AEST✓ passGLMTTFTexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2ttft_prefill_tps 483 tok/sT23
2026-10-04 22:49 AEST✓ passGLMloadexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2load_seconds 276 sT23
2026-10-04 22:48 AEST✕ crashFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2ttft_prefill_tps unavailablemeasureT14a
2026-10-04 22:47 AEST✕ crashFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailableharnessT14a
2026-10-04 22:42 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2load_seconds 257 sT14a
2026-10-04 22:32 AEST✕ crashFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_window_min_tps unavailable · tool_calls 0 callsmeasureT13
2026-10-04 22:32 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2top1_agreement 0.98967 fraction · mean_kl 0.00097327 natsT13
2026-10-04 22:30 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2ttft_prefill_tps 2,009 tok/sT13
2026-10-04 22:15 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_c1_tps 47.4 tok/s · prefill_32k_tps 2,678 tok/sT13
2026-10-04 22:10 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2lab_decode_c1_tps 55.8 tok/s · lab_prefill_tps 2,419 tok/sT13
2026-10-04 22:05 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2load_seconds 260 sT13
2026-10-04 21:27 AEST! failGLMlabexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab_decode_c1_tps 7.50 tok/s · lab_prefill_tps 720 tok/sgatesT23
2026-10-04 21:17 AEST! failGLMstabilityexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2decode_window_min_tps 3.36 tok/s · tool_calls 21 callsT23
2026-10-04 21:13 AEST! failGLMTTFTexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2ttft_prefill_tps unavailableinvalid measurementT23
2026-10-04 21:02 AEST! failGLMlabexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab_decode_c1_tps 7.40 tok/s · lab_prefill_tps 739 tok/sgatesT23
2026-10-04 20:57 AEST✓ passGLMloadexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2load_seconds 300 sT23
2026-10-04 20:56 AEST✕ crashFlash-Nextloadsglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2load_seconds unavailableloadT15
2026-10-04 20:52 AEST✕ crashFlash-Nextloadsglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2load_seconds unavailableloadT15
2026-10-04 20:47 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2lab_decode_c1_tps 54.5 tok/s · lab_prefill_tps 2,596 tok/sT13
2026-10-04 20:36 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2decode_window_min_tps 37.9 tok/s · tool_calls 67 callsT13
2026-10-04 20:36 AEST✓ passFlash-Nextquality panelsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2top1_agreement 0.98867 fraction · mean_kl 0.00096647 natsT13
2026-10-04 20:34 AEST✓ passFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2ttft_prefill_tps 2,017 tok/sT13
2026-10-04 20:19 AEST✓ passFlash-Nextkit sweepsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2decode_c1_tps 48.2 tok/s · prefill_32k_tps 2,678 tok/sT13
2026-10-04 20:14 AEST✓ passFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2lab_decode_c1_tps 55.6 tok/s · lab_prefill_tps 2,433 tok/sT13
2026-10-04 20:10 AEST✓ passFlash-Nextloadsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2load_seconds 236 sT13
2026-10-04 19:38 AEST✕ crashGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012decode_window_min_tps unavailable · tool_calls 0 callsmeasureT22c
2026-10-04 19:26 AEST✓ passGLMTTFTik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012ttft_prefill_tps 248 tok/sT22c
2026-10-04 19:12 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012lab_decode_c1_tps 10.2 tok/s · lab_prefill_tps 166 tok/sgatesT22c
2026-10-04 19:11 AEST✓ passGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012load_seconds 48.6 sT22c
2026-10-04 19:11 AEST✓ passnonehw bwhw-bw-gpu1-currenth2d_gbps 13.5 GB/s · d2h_gbps 13.2 GB/sT05a
2026-10-04 18:19 AEST✓ passFlash-NextTTFTllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012ttft_prefill_tps 980 tok/sT11
2026-10-04 18:11 AEST✓ passFlash-Nextlabllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012lab_decode_c1_tps 35.0 tok/s · lab_prefill_tps 713 tok/sT11
2026-10-04 18:10 AEST✓ passFlash-Nextloadllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012load_seconds 39.9 sT11
2026-10-04 18:06 AEST✓ passFlash-NextTTFTllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012ttft_prefill_tps 1,067 tok/sT11
2026-10-04 18:01 AEST✓ passFlash-Nextlabllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012lab_decode_c1_tps 44.3 tok/s · lab_prefill_tps 717 tok/sT11
2026-10-04 18:00 AEST✓ passFlash-Nextloadllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012load_seconds 39.9 sT11
2026-10-04 17:42 AEST✕ crashGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012load_seconds unavailableloadT22c
2026-10-04 17:31 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab_decode_c1_tps 13.4 tok/s · lab_prefill_tps 189 tok/sgatesT22c
2026-10-04 17:21 AEST! failGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012decode_window_min_tps 13.3 tok/s · tool_calls 0 callsT22c
2026-10-04 17:10 AEST✓ passGLMTTFTik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012ttft_prefill_tps 283 tok/sT22c
2026-10-04 16:58 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab_decode_c1_tps 14.0 tok/s · lab_prefill_tps 185 tok/sgatesT22c
2026-10-04 16:58 AEST✓ passGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012load_seconds 44.3 sT22c
2026-10-04 16:48 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012ttft_prefill_tps 298 tok/sT21
2026-10-04 16:30 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012lab_decode_c1_tps 11.2 tok/s · lab_prefill_tps 258 tok/sgatesT21
2026-10-04 16:30 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012load_seconds 54.9 sT21
2026-10-04 16:23 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2ttft_prefill_tps 384 tok/sT20b
2026-10-04 16:14 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2lab_decode_c1_tps 9.10 tok/s · lab_prefill_tps 370 tok/sgatesT20b
2026-10-04 16:13 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2load_seconds 48.9 sT20b
2026-10-04 16:00 AEST✓ passFlash-Nextlabllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012lab_decode_c1_tps 57.1 tok/s · lab_prefill_tps 608 tok/sT11
2026-10-04 15:49 AEST! failFlash-Nextstabilityllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012decode_window_min_tps 38.6 tok/s · tool_calls 70 callsT11
2026-10-04 15:45 AEST✓ passFlash-NextTTFTllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012ttft_prefill_tps 987 tok/sT11
2026-10-04 15:39 AEST✓ passFlash-Nextlabllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012lab_decode_c1_tps 56.9 tok/s · lab_prefill_tps 605 tok/sT11
2026-10-04 15:38 AEST✓ passFlash-Nextloadllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012load_seconds 39.9 sT11
2026-10-04 15:19 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 8.30 tok/s · lab_prefill_tps 156 tok/sgatesT21
2026-10-04 15:08 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012decode_window_min_tps 7.81 tok/s · tool_calls 25 callsT21
2026-10-04 14:52 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012ttft_prefill_tps 149 tok/sT21
2026-10-04 14:26 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 8.00 tok/s · lab_prefill_tps 139 tok/sgatesT21
2026-10-04 14:25 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012load_seconds 91.2 sT21
2026-10-04 14:24 AEST✕ crashGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012load_seconds unavailableloadT22c
2026-10-04 14:23 AEST✕ crashGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012load_seconds unavailableloadT22c
2026-10-04 14:14 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 288 tok/sgatesT20b
2026-10-04 14:04 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2decode_window_min_tps 8.85 tok/s · tool_calls 27 callsT20b
2026-10-04 13:55 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2ttft_prefill_tps 321 tok/sT20b
2026-10-04 13:42 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 288 tok/sgatesT20b
2026-10-04 13:41 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2load_seconds 45.9 sT20b
2026-10-04 13:29 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2ttft_prefill_tps 213 tok/sT20b
2026-10-04 13:06 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 201 tok/sgatesT20b
2026-10-04 13:05 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2load_seconds 45.9 sT20b
2026-10-04 12:49 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 11.8 tok/s · lab_prefill_tps 218 tok/sgatesT21
2026-10-04 12:39 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012decode_window_min_tps 11.0 tok/s · tool_calls 24 callsT21
2026-10-04 12:27 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012ttft_prefill_tps 240 tok/sT21
2026-10-04 12:16 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 11.8 tok/s · lab_prefill_tps 219 tok/sgatesT21
2026-10-04 12:15 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012load_seconds 54.9 sT21
2026-10-04 11:59 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012ttft_prefill_tps 159 tok/sT21
2026-10-04 11:39 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012lab_decode_c1_tps 12.2 tok/s · lab_prefill_tps 153 tok/sgatesT21
2026-10-04 11:38 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012load_seconds 55.0 sT21
2026-10-04 10:42 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab_decode_c1_tps 10.4 tok/s · lab_prefill_tps 132 tok/sgatesT21
2026-10-04 10:31 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012decode_window_min_tps 9.86 tok/s · tool_calls 22 callsT21
2026-10-04 10:12 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012ttft_prefill_tps 130 tok/sT21
2026-10-04 09:30 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab_decode_c1_tps 10.5 tok/s · lab_prefill_tps 126 tok/sgatesT21
2026-10-04 09:29 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012load_seconds 79.7 sT21
2026-10-04 09:17 AEST✕ crashGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012lab_decode_c1_tps unavailable · lab_prefill_tps unavailableharnessT21
2026-10-04 09:07 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012decode_window_min_tps 11.4 tok/s · tool_calls 26 callsT21
2026-10-04 08:51 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012ttft_prefill_tps 166 tok/sT21
2026-10-04 08:27 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012lab_decode_c1_tps 12.3 tok/s · lab_prefill_tps 158 tok/sgatesT21
2026-10-04 08:25 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012load_seconds 101 sT21
2026-10-04 08:17 AEST! failFlash-Nextstabilityllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012decode_window_min_tps unavailable · tool_calls 0 callsT11
2026-10-04 08:06 AEST✓ passFlash-Nextloadllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012load_seconds 39.9 sT11
2026-10-04 06:02 AEST✕ crashGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2decode_window_min_tps unavailable · tool_calls 0 callsmeasureT22b
2026-10-04 05:48 AEST✓ passGLMTTFTik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2ttft_prefill_tps 199 tok/sT22b
2026-10-04 05:33 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2lab_decode_c1_tps 8.60 tok/s · lab_prefill_tps 147 tok/sgatesT22b
2026-10-04 05:32 AEST✓ passGLMloadik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2load_seconds 51.9 sT22b
2026-10-04 05:16 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 199 tok/sgatesT20b
2026-10-04 05:06 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2decode_window_min_tps 8.85 tok/s · tool_calls 30 callsT20b
2026-10-04 04:54 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2ttft_prefill_tps 212 tok/sT20b
2026-10-04 04:26 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 200 tok/sgatesT20b
2026-10-04 04:26 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2load_seconds 45.9 sT20b
2026-10-04 03:40 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab_decode_c1_tps 8.40 tok/s · lab_prefill_tps 71 tok/sgatesT20b
2026-10-04 03:30 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2decode_window_min_tps 7.65 tok/s · tool_calls 25 callsT20b
2026-10-04 02:55 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2ttft_prefill_tps 75.9 tok/sT20b
2026-10-04 02:26 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab_decode_c1_tps 8.30 tok/s · lab_prefill_tps 65 tok/sgatesT20b
2026-10-04 02:25 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2load_seconds 58.2 sT20b
2026-10-04 01:51 AEST✕ crashGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2decode_window_min_tps unavailable · tool_calls 0 callsmeasureT22b
2026-10-04 01:37 AEST✓ passGLMTTFTik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2ttft_prefill_tps 201 tok/sT22b
2026-10-04 01:19 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2lab_decode_c1_tps 8.20 tok/s · lab_prefill_tps 147 tok/sgatesT22b
2026-10-04 01:18 AEST✓ passGLMloadik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2load_seconds 47.7 sT22b
2026-10-04 01:01 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 151 tok/sgatesT22b
2026-10-04 00:51 AEST! failGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2decode_window_min_tps 9.46 tok/s · tool_calls 0 callsT22b
2026-10-04 00:37 AEST✓ passGLMTTFTik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2ttft_prefill_tps 203 tok/sT22b
2026-10-04 00:22 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab_decode_c1_tps 10.0 tok/s · lab_prefill_tps 151 tok/sgatesT22b
2026-10-04 00:22 AEST✓ passGLMloadik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2load_seconds 46.1 sT22b
2026-10-03 23:31 AEST! failGLMlabllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab_decode_c1_tps 9.30 tok/s · lab_prefill_tps 80 tok/sgatesT20b
2026-10-03 23:21 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2decode_window_min_tps 8.80 tok/s · tool_calls 19 callsT20b
2026-10-03 22:50 AEST✓ passGLMTTFTllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2ttft_prefill_tps 84.4 tok/sT20b
2026-10-03 21:59 AEST! failGLMlabllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab_decode_c1_tps 9.20 tok/s · lab_prefill_tps 78 tok/sgatesT20b
2026-10-03 21:58 AEST✓ passGLMloadllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2load_seconds 49.0 sT20b
2026-10-03 12:48 AEST! failGLMlabglm-llamacpp-iq2m-128k-1x-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 81 tok/sgatesT20b
2026-10-03 12:38 AEST! failGLMstabilityglm-llamacpp-iq2m-128k-1x-gpu2decode_window_min_tps 8.91 tok/s · tool_calls 24 callsT20b
2026-10-03 12:07 AEST✓ passGLMTTFTglm-llamacpp-iq2m-128k-1x-gpu2ttft_prefill_tps 86.3 tok/sT20b
2026-10-03 11:41 AEST! failGLMlabglm-llamacpp-iq2m-128k-1x-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 81 tok/sgatesT20b
2026-10-03 11:41 AEST✓ passGLMloadglm-llamacpp-iq2m-128k-1x-gpu2load_seconds 45.9 sT20b
2026-10-03 11:14 AEST✓ passFlash-Nextlabfn-llamacpp-iq4xs-128k-1x-fit-gpu2lab_decode_c1_tps 26.1 tok/s · lab_prefill_tps 157 tok/sT11
2026-10-03 11:03 AEST! failFlash-Nextstabilityfn-llamacpp-iq4xs-128k-1x-fit-gpu2decode_window_min_tps 23.8 tok/s · tool_calls 49 callsT11
2026-10-03 10:49 AEST✓ passFlash-NextTTFTfn-llamacpp-iq4xs-128k-1x-fit-gpu2ttft_prefill_tps 269 tok/sT11
2026-10-03 10:29 AEST✓ passFlash-Nextlabfn-llamacpp-iq4xs-128k-1x-fit-gpu2lab_decode_c1_tps 26.5 tok/s · lab_prefill_tps 157 tok/sT11
2026-10-03 10:29 AEST✓ passFlash-Nextloadfn-llamacpp-iq4xs-128k-1x-fit-gpu2load_seconds 27.8 sT11
2026-10-03 10:28 AEST✕ crashFlash-Nextloadfn-llamacpp-iq4xs-128k-1x-gpu2load_seconds unavailableloadT11
2026-10-03 08:17 AEST✓ passFlash-Nextlabfn-trellis-3.05-0xsero-main-x8-gpu2lab_decode_c1_tps 55.3 tok/s · lab_prefill_tps 2,228 tok/sT10
2026-10-03 08:06 AEST! failFlash-Nextstabilityfn-trellis-3.05-0xsero-main-x8-gpu2decode_window_min_tps 38.1 tok/s · tool_calls 63 callsT10
2026-10-03 08:06 AEST✓ passFlash-Nextquality panelfn-trellis-3.05-0xsero-main-x8-gpu2top1_agreement 0.99234 fraction · mean_kl 0.0009667 natsT10
2026-10-03 08:04 AEST✓ passFlash-NextTTFTfn-trellis-3.05-0xsero-main-x8-gpu2ttft_prefill_tps 2,263 tok/sT10
2026-10-03 07:48 AEST✓ passFlash-Nextkit sweepfn-trellis-3.05-0xsero-main-x8-gpu2decode_c1_tps 46.9 tok/s · prefill_32k_tps 2,433 tok/sT10
2026-10-03 07:43 AEST✓ passFlash-Nextlabfn-trellis-3.05-0xsero-main-x8-gpu2lab_decode_c1_tps 49.3 tok/s · lab_prefill_tps 2,176 tok/sT10
2026-10-03 07:40 AEST✓ passFlash-Nextloadfn-trellis-3.05-0xsero-main-x8-gpu2load_seconds 179 sT10
2026-10-03 06:54 AEST✓ passnonenvmenvme-rand-1tb-ngramrand_read_iops 168,600 IOPS · rand_read_gbps 3.35 GB/sT05a
2026-10-03 06:53 AEST✓ passnonehw bwhw-bw-gpu0-currenth2d_gbps 6.03 GB/s · d2h_gbps 6.60 GB/sT05a
2026-10-03 06:53 AEST✓ passnonehw bwhw-bw-gpu2-currenth2d_gbps 13.5 GB/s · d2h_gbps 13.2 GB/sT05a

Runs / none

20261002T205342Z-hw-bw-gpu2-current-hw_bw

✓ pass hw bw T05a eligibility: none

Confighw-bw-gpu2-current
Started2026-10-03 06:53 AEST
Finished2026-10-03 06:53 AEST (0.1 min)
Exit status0
Notes1 GiB pinned transfers; full table in detail

Launch

no container: static measurement

Metrics

context.configured–
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
h2d_gbps13.5 GB/s {"1MB_pin": 13.15, "1MB_page": 13.02, "4MB_pin": 13.39, "4MB_page": 13.14, "16MB_pin": 13.45, "16MB_page": 13.19, "64MB_pin": 13.46, "64MB_page": 13.18, "256MB_pin": 13.47, "256MB_page": 13.16, "1024MB_pin": 13.47, "1024MB_page": 13.11}
d2h_gbps13.2 GB/s {"1MB_pin": 12.98, "1MB_page": 7.19, "4MB_pin": 13.14, "4MB_page": 10.06, "16MB_pin": 13.2, "16MB_page": 11.35, "64MB_pin": 13.21, "64MB_page": 12.57, "256MB_pin": 13.21, "256MB_page": 12.61, "1024MB_pin": 13.22, "1024MB_page": 12.69}
host_memcpy_gbps16.6 GB/s

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit483c32c7f11d39f2f1e84c87602db3e166dc9fff clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/none/20261002T205342Z-hw-bw-gpu2-current-hw_bw.json

Runs / none

20261002T205350Z-hw-bw-gpu0-current-hw_bw

✓ pass hw bw T05a eligibility: none

Confighw-bw-gpu0-current
Started2026-10-03 06:53 AEST
Finished2026-10-03 06:54 AEST (0.2 min)
Exit status0
Notes1 GiB pinned transfers; full table in detail

Launch

no container: static measurement

Metrics

context.configured–
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
h2d_gbps6.03 GB/s {"1MB_pin": 6.48, "1MB_page": 6.08, "4MB_pin": 6.6, "4MB_page": 6.1, "16MB_pin": 6.57, "16MB_page": 6.11, "64MB_pin": 6.09, "64MB_page": 6.05, "256MB_pin": 6.03, "256MB_page": 6.01, "1024MB_pin": 6.03, "1024MB_page": 5.99}
d2h_gbps6.60 GB/s {"1MB_pin": 6.51, "1MB_page": 4.62, "4MB_pin": 6.58, "4MB_page": 5.88, "16MB_pin": 6.59, "16MB_page": 6.34, "64MB_pin": 6.6, "64MB_page": 6.48, "256MB_pin": 6.6, "256MB_page": 6.51, "1024MB_pin": 6.6, "1024MB_page": 6.52}
host_memcpy_gbps16.6 GB/s

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit483c32c7f11d39f2f1e84c87602db3e166dc9fff clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/none/20261002T205350Z-hw-bw-gpu0-current-hw_bw.json

Runs / none

20261002T205409Z-nvme-rand-1tb-ngram-nvme_rand

✓ pass nvme T05a eligibility: none

Confignvme-rand-1tb-ngram
Started2026-10-03 06:54 AEST
Finished2026-10-03 06:55 AEST (1.1 min)
Exit status0
Notes~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5/ngram_embedding.safetensors

Launch

no container: static measurement

Metrics

context.configured–
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
rand_read_iops168,600 IOPS "16 KiB, 32 threads"
rand_read_gbps3.35 GB/s [{"bs": 4096, "threads": 1, "GBps": 0.075, "kIOPS": 18.2}, {"bs": 4096, "threads": 8, "GBps": 0.502, "kIOPS": 122.6}, {"bs": 4096, "threads": 32, "GBps": 0.714, "kIOPS": 174.4}, {"bs": 4096, "threads": 64, "GBps": 0.697, "kIOPS": 170.3}, {"bs": 16384, "threads": 1, "GBps": 0.2, "kIOPS": 12.2}, {"bs": 16384, "threads": 8, "GBps": 1.328, "kIOPS": 81.1}, {"bs": 16384, "threads": 32, "GBps": 2.763, "kIOPS": 168.6}, {"bs": 16384, "threads": 64, "GBps": 2.678, "kIOPS": 163.4}, {"bs": 65536, "threads": 1, "GBps": 0.47, "kIOPS": 7.2}, {"bs": 65536, "threads": 8, "GBps": 2.96, "kIOPS": 45.2}, {"bs": 65536, "threads": 32, "GBps": 3.35, "kIOPS": 51.1}, {"bs": 65536, "threads": 64, "GBps": 3.35, "kIOPS": 51.1}, {"bs": 1048576, "threads": 1, "GBps": 1.972, "kIOPS": 1.9}, {"bs": 1048576, "threads": 8, "GBps": 3.35, "kIOPS": 3.2}, {"bs": 1048576, "threads": 32, "GBps": 3.353, "kIOPS": 3.2}, {"bs": 1048576, "threads": 64, "GBps": 3.355, "kIOPS": 3.2}]

Cache, layout and host

Cachewarm
Layoutcurrent · 0 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit483c32c7f11d39f2f1e84c87602db3e166dc9fff clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/none/20261002T205409Z-nvme-rand-1tb-ngram-nvme_rand.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T214031Z-fn-trellis-3.05-0xsero-main-x8-gpu2-load

✓ pass load T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 07:40 AEST
Finished2026-10-03 07:43 AEST (3.0 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds179 s
host_ram_drop_gb63.7 GB
vram_ready_mib21,060 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T214031Z-fn-trellis-3.05-0xsero-main-x8-gpu2-load.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T214330Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab

✓ pass lab T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 07:43 AEST
Finished2026-10-03 07:48 AEST (5.3 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps49.3 tok/s
lab_prefill_tps2,176 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T214330Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T214848Z-fn-trellis-3.05-0xsero-main-x8-gpu2-kit_sweep

✓ pass kit sweep T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 07:48 AEST
Finished2026-10-03 08:04 AEST (15.9 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,292 tok/s {"min": 2277.9, "max": 2305.2, "n": 3}
prefill_16k_tps2,401 tok/s {"min": 2400.6, "max": 2403.1, "n": 3}
prefill_32k_tps2,433 tok/s {"min": 2431.6, "max": 2433.5, "n": 3}
prefill_64k_tps2,415 tok/s {"min": 2413.8, "max": 2415.0, "n": 3}
decode_c1_tps46.9 tok/s {"min": 45.1, "max": 48.7, "power_w": null}
decode_c2_tps51.6 tok/s {"min": 51.1, "max": 52.11, "power_w": null}
decode_c3_tps53.9 tok/s {"min": 51.82, "max": 55.98, "power_w": null}
decode_c4_tps51.8 tok/s {"min": 50.57, "max": 53.12, "power_w": null}
decode_c1_32k_tps48.5 tok/s {"min": 48.51, "max": 48.6, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T214848Z-fn-trellis-3.05-0xsero-main-x8-gpu2-kit_sweep.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T220441Z-fn-trellis-3.05-0xsero-main-x8-gpu2-ttft

✓ pass TTFT T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 08:04 AEST
Finished2026-10-03 08:06 AEST (1.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,263 tok/s {"prompt_tokens": [13679, 13707, 13790], "ttft_s": [6.049, 6.056, 6.045]}
ttft_prefill_32k_tps2,368 tok/s {"prompt_tokens": [54780, 54797, 54620], "ttft_s": [23.132, 23.12, 23.139]}
ttft_prefill_tps2,263 tok/s {"prompt_tokens": [13679, 13707, 13790], "ttft_s": [6.049, 6.056, 6.045]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T220441Z-fn-trellis-3.05-0xsero-main-x8-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T220609Z-fn-trellis-3.05-0xsero-main-x8-gpu2-quality_panel

✓ pass quality panel T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 08:06 AEST
Finished2026-10-03 08:06 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.99234 fraction
mean_kl0.0009667 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T220609Z-fn-trellis-3.05-0xsero-main-x8-gpu2-quality_panel.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T220633Z-fn-trellis-3.05-0xsero-main-x8-gpu2-stability

! fail stability T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 08:06 AEST
Finished2026-10-03 08:17 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 63 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.077 < floor 50.0

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max30,034 tokens
context.headroom_min174,233 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls63 calls {"per_min": 6.3, "requests": 45}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps38.1 tok/s
window 1: 42.97 tok/swindow 2: 43.5 tok/swindow 3: 47.28 tok/swindow 4: 41.43 tok/swindow 5: 43.59 tok/swindow 6: 42.67 tok/smin 41.43 · max 47.28 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 331, "min_at_generation_s": 63.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.1 tok/s {"generation_seconds": 450.481, "generated_tokens": 19411.4}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 208.0, "requests": 18, "tool_calls": 26, "completion_tokens": 6387, "occupied_max": 12502}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 392.0, "requests": 27, "tool_calls": 37, "completion_tokens": 13133, "occupied_max": 30034}] list
tasks_over_64k[] instance ids
completion_tokens19,520 tokens {"finish_reasons": {"tool_calls": 45}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 63 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 38.1 tok/s, whole run 43.1 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T220633Z-fn-trellis-3.05-0xsero-main-x8-gpu2-stability.json

Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2

20261002T221703Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab

✓ pass lab T10 eligibility: none

Configfn-trellis-3.05-0xsero-main-x8-gpu2
Started2026-10-03 08:17 AEST
Finished2026-10-03 08:22 AEST (5.0 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt)
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.3 tok/s
lab_prefill_tps2,228 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit966c1355dde3774c7406f9d18e6b8468d4a10fdb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261002T221703Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-gpu2

20261003T002815Z-fn-llamacpp-iq4xs-128k-1x-gpu2-load

✕ crash load T11 eligibility: none failed stage: load

Configfn-llamacpp-iq4xs-128k-1x-gpu2
Started2026-10-03 10:28 AEST
Finished2026-10-03 10:28 AEST (0.4 min)
Exit status1
Notesuser, abort 0.18.994.031 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 65358.17 MiB on device 0: cudaMalloc failed: out of memory 0.18.994.037 E alloc_tensor_range: failed to allocate CUDA0 buffer of size 68533006336 0.19.139.695 E llama_model_load: error loading model: unable to allocate CUDA0 buffer 0.19.139.702 E llama_model_load_from_file_impl: failed to load model 0.19.139.708 E cmn common_init_: failed to load model '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf' 0.19.139.713 E srv load_model: failed to load model, '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf' 0.19.139.715 I srv operator(): operator(): cleaning up before exit... 0.19.140.570 E srv llama_server: exiting due to model loading error

Launch

recorded by tools/run.py (artifacts/runs/20261003T002815Z-lab-fn-llamacpp-iq4xs-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 -ot per_layer_token_embd.weight=CPU
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x.json sha256 697275c75292e4ef45a02a99b2a98c23b7a55630ed0430af0972a1dcf84b47d8
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 -ot per_layer_token_embd.weight=CPU
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_secondsunavailable not produced (crash/load)
host_ram_drop_gbunavailable not produced (crash/load)
vram_ready_mibunavailable not produced (crash/load)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1fa796f9d509705f24dc1870357869cf7c293bc2 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T002815Z-fn-llamacpp-iq4xs-128k-1x-gpu2-load.json

Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2

20261003T002901Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-load

✓ pass load T11 eligibility: none

Configfn-llamacpp-iq4xs-128k-1x-fit-gpu2
Started2026-10-03 10:29 AEST
Finished2026-10-03 10:29 AEST (0.5 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt)
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds27.8 s
host_ram_drop_gb2.30 GB
vram_ready_mib22,684 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T002901Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-load.json

Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2

20261003T002929Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab

✓ pass lab T11 eligibility: none

Configfn-llamacpp-iq4xs-128k-1x-fit-gpu2
Started2026-10-03 10:29 AEST
Finished2026-10-03 10:49 AEST (19.9 min)
Exit status0
Notesserved=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt)
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps26.5 tok/s
lab_prefill_tps157 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T002929Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2

20261003T004923Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-ttft

✓ pass TTFT T11 eligibility: none

Configfn-llamacpp-iq4xs-128k-1x-fit-gpu2
Started2026-10-03 10:49 AEST
Finished2026-10-03 11:03 AEST (14.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt)
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps269 tok/s {"prompt_tokens": [13827, 13626, 13781], "ttft_s": [50.985, 51.655, 51.198]}
ttft_prefill_32k_tps230 tok/s {"prompt_tokens": [54692, 54711, 54604], "ttft_s": [237.983, 237.379, 240.167]}
ttft_prefill_tps269 tok/s {"prompt_tokens": [13827, 13626, 13781], "ttft_s": [50.985, 51.655, 51.198]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T004923Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2

20261003T010352Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-stability

! fail stability T11 eligibility: none

Configfn-llamacpp-iq4xs-128k-1x-fit-gpu2
Started2026-10-03 11:03 AEST
Finished2026-10-03 11:14 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 49 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 23.83 < floor 50.0

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt)
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max16,819 tokens
context.headroom_min114,087 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls49 calls {"per_min": 4.9, "requests": 32}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps23.8 tok/s
window 1: 24.81 tok/swindow 2: 24.06 tok/swindow 3: 25.42 tok/swindow 4: 24.9 tok/swindow 5: 24.6 tok/smin 24.06 · max 25.42 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 271, "min_at_generation_s": 105.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps24.8 tok/s {"generation_seconds": 390.845, "generated_tokens": 9687.8}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 306.9, "requests": 17, "tool_calls": 26, "completion_tokens": 4940, "occupied_max": 16819}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 293.1, "requests": 16, "tool_calls": 24, "completion_tokens": 4788, "occupied_max": 12213}] list
tasks_over_64k[] instance ids
completion_tokens9,728 tokens {"finish_reasons": {"tool_calls": 32}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 49 tool calls, 0 parse failures, 1 repetition hits, tasks solved 1/2, decode min window 23.8 tok/s, whole run 24.8 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T010352Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-stability.json

Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2

20261003T011420Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab

✓ pass lab T11 eligibility: none

Configfn-llamacpp-iq4xs-128k-1x-fit-gpu2
Started2026-10-03 11:14 AEST
Finished2026-10-03 11:40 AEST (26.5 min)
Exit status0
Notesserved=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt)
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps26.1 tok/s
lab_prefill_tps157 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T011420Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab.json

Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2

20261003T014103Z-glm-llamacpp-iq2m-128k-1x-gpu2-load

✓ pass load T20b eligibility: none

Configglm-llamacpp-iq2m-128k-1x-gpu2
Started2026-10-03 11:41 AEST
Finished2026-10-03 11:41 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds45.9 s
host_ram_drop_gb2.23 GB
vram_ready_mib22,784 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit4b12eecb28cf351fc09677fc509debe454394308 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T014103Z-glm-llamacpp-iq2m-128k-1x-gpu2-load.json

Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2

20261003T014149Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configglm-llamacpp-iq2m-128k-1x-gpu2
Started2026-10-03 11:41 AEST
Finished2026-10-03 12:07 AEST (26.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.60 tok/s
lab_prefill_tps81 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit4b12eecb28cf351fc09677fc509debe454394308 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T014149Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2

20261003T020751Z-glm-llamacpp-iq2m-128k-1x-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configglm-llamacpp-iq2m-128k-1x-gpu2
Started2026-10-03 12:07 AEST
Finished2026-10-03 12:38 AEST (30.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps86.3 tok/s {"prompt_tokens": [10742, 10816, 10721], "ttft_s": [124.451, 126.526, 123.598]}
ttft_prefill_32k_tps88.5 tok/s {"prompt_tokens": [42736, 42800, 42830], "ttft_s": [485.781, 483.408, 483.895]}
ttft_prefill_tps86.3 tok/s {"prompt_tokens": [10742, 10816, 10721], "ttft_s": [124.451, 126.526, 123.598]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbff973e82a9b895daba523707af9c56454e76a39 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T020751Z-glm-llamacpp-iq2m-128k-1x-gpu2-ttft.json

Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2

20261003T023819Z-glm-llamacpp-iq2m-128k-1x-gpu2-stability

! fail stability T20b eligibility: none

Configglm-llamacpp-iq2m-128k-1x-gpu2
Started2026-10-03 12:38 AEST
Finished2026-10-03 12:48 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.91 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max9,047 tokens
context.headroom_min121,785 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls24 calls {"per_min": 2.4, "requests": 13}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps8.91 tok/s
window 1: 8.98 tok/swindow 2: 9.23 tok/swindow 3: 9.09 tok/swindow 4: 9.18 tok/swindow 5: 9.04 tok/swindow 6: 9.14 tok/smin 8.98 · max 9.23 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 328, "min_at_generation_s": 90.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps9.14 tok/s {"generation_seconds": 447.593, "generated_tokens": 4091.5}
tasks_attempted1 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 600.0, "requests": 14, "tool_calls": 24, "completion_tokens": 4105, "occupied_max": 9047}] list
tasks_over_64k[] instance ids
completion_tokens4,105 tokens {"finish_reasons": {"tool_calls": 13}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 24 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/1, decode min window 8.91 tok/s, whole run 9.14 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbff973e82a9b895daba523707af9c56454e76a39 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T023819Z-glm-llamacpp-iq2m-128k-1x-gpu2-stability.json

Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2

20261003T024824Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configglm-llamacpp-iq2m-128k-1x-gpu2
Started2026-10-03 12:48 AEST
Finished2026-10-03 13:14 AEST (25.9 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.60 tok/s
lab_prefill_tps81 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbff973e82a9b895daba523707af9c56454e76a39 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T024824Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2

20261003T115813Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-load

✓ pass load T20b eligibility: none

Configllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
Started2026-10-03 21:58 AEST
Finished2026-10-03 21:59 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds49.0 s
host_ram_drop_gb2.22 GB
vram_ready_mib22,626 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit73bf82fd3254285cf46014dbef7e02e6ef6db9b3 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T115813Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2

20261003T115902Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
Started2026-10-03 21:59 AEST
Finished2026-10-03 22:50 AEST (51.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.20 tok/s
lab_prefill_tps78 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbc100ca481d268577db82a7adcd92133db2241e4 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T115902Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2

20261003T125002Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
Started2026-10-03 22:50 AEST
Finished2026-10-03 23:21 AEST (31.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps84.4 tok/s {"prompt_tokens": [10644, 10721, 10688], "ttft_s": [126.044, 127.442, 125.735]}
ttft_prefill_32k_tps84.9 tok/s {"prompt_tokens": [42901, 42759, 42872], "ttft_s": [504.114, 504.375, 505.138]}
ttft_prefill_tps84.4 tok/s {"prompt_tokens": [10644, 10721, 10688], "ttft_s": [126.044, 127.442, 125.735]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbc100ca481d268577db82a7adcd92133db2241e4 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T125002Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2

20261003T132135Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-stability

! fail stability T20b eligibility: none

Configllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
Started2026-10-03 23:21 AEST
Finished2026-10-03 23:31 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 19 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.8 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json

Metrics

context.configured131,072 tokens
context.occupied_max6,658 tokens
context.headroom_min123,658 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls19 calls {"per_min": 1.9, "requests": 11}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps8.80 tok/s
window 1: 8.89 tok/swindow 2: 8.87 tok/swindow 3: 8.93 tok/swindow 4: 8.88 tok/swindow 5: 9.0 tok/smin 8.87 · max 9 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 267, "min_at_generation_s": 131.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps8.93 tok/s {"generation_seconds": 386.503, "generated_tokens": 3451.5}
tasks_attempted1 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 600.0, "requests": 12, "tool_calls": 20, "completion_tokens": 3463, "occupied_max": 6658}] list
tasks_over_64k[] instance ids
completion_tokens3,463 tokens {"finish_reasons": {"tool_calls": 11}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 19 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/1, decode min window 8.80 tok/s, whole run 8.93 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbc100ca481d268577db82a7adcd92133db2241e4 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T132135Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2

20261003T133139Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
Started2026-10-03 23:31 AEST
Finished2026-10-04 00:21 AEST (50.3 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.30 tok/s
lab_prefill_tps80 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbc100ca481d268577db82a7adcd92133db2241e4 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T133139Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2

20261003T142203Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-load

✓ pass load T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
Started2026-10-04 00:22 AEST
Finished2026-10-04 00:22 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44e
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds46.1 s
host_ram_drop_gb111 GB
vram_ready_mib14,936 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit57320e8fbf776bca99fc63e2de1647d22ef83f06 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T142203Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-load.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2

20261003T142250Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab

! fail lab T22b eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
Started2026-10-04 00:22 AEST
Finished2026-10-04 00:37 AEST (14.7 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44e
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps10.0 tok/s
lab_prefill_tps151 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit57320e8fbf776bca99fc63e2de1647d22ef83f06 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T142250Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2

20261003T143733Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-ttft

✓ pass TTFT T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
Started2026-10-04 00:37 AEST
Finished2026-10-04 00:51 AEST (13.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44e
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps203 tok/s {"prompt_tokens": [10751, 10785, 10716], "ttft_s": [52.615, 53.39, 52.749]}
ttft_prefill_32k_tps197 tok/s {"prompt_tokens": [42739, 42927, 42933], "ttft_s": [217.584, 218.259, 218.191]}
ttft_prefill_tps203 tok/s {"prompt_tokens": [10751, 10785, 10716], "ttft_s": [52.615, 53.39, 52.749]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit57320e8fbf776bca99fc63e2de1647d22ef83f06 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T143733Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-ttft.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2

20261003T145106Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-stability

! fail stability T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
Started2026-10-04 00:51 AEST
Finished2026-10-04 01:01 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 47 tool-call parse failure(s); decode window min 9.458 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44e
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max1,874 tokens
context.headroom_min129,125 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls0 calls {"per_min": 0.0, "requests": 47}
parse_failures47 responses {"wire": {"unparsed_markup": 47}, "harness_format_errors": {"no_tool_call": 47}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors47 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps9.46 tok/s
window 1: 9.49 tok/swindow 2: 9.55 tok/swindow 3: 9.5 tok/swindow 4: 9.46 tok/swindow 5: 9.55 tok/smin 9.46 · max 9.55 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 277, "min_at_generation_s": 144.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps9.50 tok/s {"generation_seconds": 396.975, "generated_tokens": 3772.8}
tasks_attempted16 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 51.1, "requests": 3, "tool_calls": 0, "completion_tokens": 310, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 40.8, "requests": 3, "tool_calls": 0, "completion_tokens": 227, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 39.7, "requests": 3, "tool_calls": 0, "completion_tokens": 220, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 43.5, "requests": 3, "tool_calls": 0, "completion_tokens": 251, "occupied_max": 1874}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 41.3, "requests": 3, "tool_calls": 0, "completion_tokens": 233, "occupied_max": 1770}, {"attempt": 6, "round": 1, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 34.9, "requests": 3, "tool_calls": 0, "completion_tokens": 174, "occupied_max": 1716}, {"attempt": 7, "round": 1, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 41.1, "requests": 3, "tool_calls": 0, "completion_tokens": 231, "occupied_max": 1641}, {"attempt": 8, "round": 1, "instance_id": "sympy__sympy-20590", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 52.5, "requests": 3, "tool_calls": 0, "completion_tokens": 339, "occupied_max": 1578}, {"attempt": 9, "round": 2, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 31.5, "requests": 3, "tool_calls": 0, "completion_tokens": 230, "occupied_max": 1559}, {"attempt": 10, "round": 2, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 35.7, "requests": 3, "tool_calls": 0, "completion_tokens": 270, "occupied_max": 1641}, {"attempt": 11, "round": 2, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 32.8, "requests": 3, "tool_calls": 0, "completion_tokens": 243, "occupied_max": 1609}, {"attempt": 12, "round": 2, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 33.5, "requests": 3, "tool_calls": 0, "completion_tokens": 249, "occupied_max": 1874}, {"attempt": 13, "round": 2, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 33.7, "requests": 3, "tool_calls": 0, "completion_tokens": 251, "occupied_max": 1770}, {"attempt": 14, "round": 2, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.9, "requests": 3, "tool_calls": 0, "completion_tokens": 178, "occupied_max": 1716}, {"attempt": 15, "round": 2, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 30.0, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1641}, {"attempt": 16, "round": 2, "instance_id": "sympy__sympy-20590", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 32.0, "requests": 3, "tool_calls": 0, "completion_tokens": 198, "occupied_max": 1459}] list
tasks_over_64k[] instance ids
completion_tokens3,821 tokens {"finish_reasons": {"stop": 47}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 0 tool calls, 47 parse failures, 0 repetition hits, tasks solved 0/16, decode min window 9.46 tok/s, whole run 9.50 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit57320e8fbf776bca99fc63e2de1647d22ef83f06 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T145106Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-stability.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2

20261003T150111Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab

! fail lab T22b eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
Started2026-10-04 01:01 AEST
Finished2026-10-04 01:18 AEST (17.7 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44e
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps9.50 tok/s
lab_prefill_tps151 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit57320e8fbf776bca99fc63e2de1647d22ef83f06 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T150111Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2

20261003T151858Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-load

✓ pass load T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
Started2026-10-04 01:18 AEST
Finished2026-10-04 01:19 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds47.7 s
host_ram_drop_gb115 GB
vram_ready_mib22,318 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit664783f004ff58b6fcabb0691d6b1c8298e2e818 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T151858Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-load.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2

20261003T151947Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-lab

! fail lab T22b eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
Started2026-10-04 01:19 AEST
Finished2026-10-04 01:37 AEST (17.7 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps8.20 tok/s
lab_prefill_tps147 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsunavailable not produced (fail/gates)
mtp_acceptance_rateunavailable not produced (fail/gates)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit937eb11054a0e7abd1b0e970f25ea2fda3f4cff9 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T151947Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2

20261003T153729Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-ttft

✓ pass TTFT T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
Started2026-10-04 01:37 AEST
Finished2026-10-04 01:51 AEST (13.9 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps201 tok/s {"prompt_tokens": [10740, 10677, 10728], "ttft_s": [53.545, 52.784, 53.644]}
ttft_prefill_32k_tps190 tok/s {"prompt_tokens": [42815, 42706, 42534], "ttft_s": [223.459, 224.655, 228.199]}
ttft_prefill_tps201 tok/s {"prompt_tokens": [10740, 10677, 10728], "ttft_s": [53.545, 52.784, 53.644]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit937eb11054a0e7abd1b0e970f25ea2fda3f4cff9 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T153729Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-ttft.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2

20261003T155126Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-stability

✕ crash stability T22b eligibility: none failed stage: measure

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
Started2026-10-04 01:51 AEST
Finished2026-10-04 02:01 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max1,641 tokens
context.headroom_min129,356 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls0 calls {"per_min": 0.0, "requests": 9}
parse_failures9 responses {"wire": {"unparsed_markup": 9}, "harness_format_errors": {"no_tool_call": 9}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors9 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tpsunavailable only 57.72 s of generation; need > 120 s for one window
decode_whole_run_tps12.6 tok/s {"generation_seconds": 57.72, "generated_tokens": 729.8}
tasks_attempted5 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomyes {"upstream_connection_failures": 54, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 47.1, "requests": 3, "tool_calls": 0, "completion_tokens": 274, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 36.4, "requests": 3, "tool_calls": 0, "completion_tokens": 248, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 33.8, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 343.5}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 139.3}] list
tasks_over_64k[] instance ids
completion_tokens739 tokens {"finish_reasons": {"stop": 9}}
mtp_accepted_tpsunavailable not produced (crash/measure)
mtp_acceptance_rateunavailable not produced (crash/measure)

Agentic summary: 600 s, 0 tool calls, 9 parse failures, 0 repetition hits, tasks solved 0/5, decode min window – tok/s, whole run 12.6 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit937eb11054a0e7abd1b0e970f25ea2fda3f4cff9 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T155126Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2

20261003T162532Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-load

✓ pass load T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
Started2026-10-04 02:25 AEST
Finished2026-10-04 02:26 AEST (1.0 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds58.2 s
host_ram_drop_gb1.89 GB
vram_ready_mib22,596 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9fc82ec3597413063d86252387463dee533769ed clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T162532Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2

20261003T162631Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
Started2026-10-04 02:26 AEST
Finished2026-10-04 02:55 AEST (28.7 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps8.30 tok/s
lab_prefill_tps65 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9fc82ec3597413063d86252387463dee533769ed clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T162631Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2

20261003T165514Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
Started2026-10-04 02:55 AEST
Finished2026-10-04 03:30 AEST (34.8 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps75.9 tok/s {"prompt_tokens": [10736, 10754, 10667], "ttft_s": [140.833, 141.619, 141.761]}
ttft_prefill_32k_tps77.3 tok/s {"prompt_tokens": [42840, 42803, 42831], "ttft_s": [554.48, 553.698, 555.603]}
ttft_prefill_tps75.9 tok/s {"prompt_tokens": [10736, 10754, 10667], "ttft_s": [140.833, 141.619, 141.761]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9fc82ec3597413063d86252387463dee533769ed clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T165514Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2

20261003T173003Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-stability

! fail stability T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
Started2026-10-04 03:30 AEST
Finished2026-10-04 03:40 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.646 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_max4,875 tokens
context.headroom_min126,123 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls25 calls {"per_min": 2.5, "requests": 15}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps7.65 tok/s
window 1: 8.08 tok/swindow 2: 7.78 tok/swindow 3: 7.99 tok/swindow 4: 8.16 tok/swindow 5: 8.19 tok/smin 7.78 · max 8.19 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 269, "min_at_generation_s": 156.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps8.08 tok/s {"generation_seconds": 388.925, "generated_tokens": 3143.9}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 377.0, "requests": 10, "tool_calls": 17, "completion_tokens": 2133, "occupied_max": 4875}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 223.0, "requests": 6, "tool_calls": 9, "completion_tokens": 1027, "occupied_max": 4558}] list
tasks_over_64k[] instance ids
completion_tokens3,160 tokens {"finish_reasons": {"tool_calls": 15}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 25 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 7.65 tok/s, whole run 8.08 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9fc82ec3597413063d86252387463dee533769ed clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T173003Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2

20261003T174033Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
Started2026-10-04 03:40 AEST
Finished2026-10-04 04:25 AEST (45.4 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps8.40 tok/s
lab_prefill_tps71 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9fc82ec3597413063d86252387463dee533769ed clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T174033Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2

20261003T182602Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-load

✓ pass load T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
Started2026-10-04 04:26 AEST
Finished2026-10-04 04:26 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94fe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds45.9 s
host_ram_drop_gb3.54 GB
vram_ready_mib21,944 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd4f093c724ebfd683feed823be35a11d4c009375 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T182602Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2

20261003T182648Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
Started2026-10-04 04:26 AEST
Finished2026-10-04 04:54 AEST (27.5 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94fe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.50 tok/s
lab_prefill_tps200 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd4f093c724ebfd683feed823be35a11d4c009375 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T182648Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2

20261003T185415Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
Started2026-10-04 04:54 AEST
Finished2026-10-04 05:06 AEST (12.2 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94fe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps212 tok/s {"prompt_tokens": [10787, 10703, 10755], "ttft_s": [50.583, 51.91, 50.672]}
ttft_prefill_32k_tps221 tok/s {"prompt_tokens": [42782, 42887, 42759], "ttft_s": [192.559, 193.833, 193.907]}
ttft_prefill_tps212 tok/s {"prompt_tokens": [10787, 10703, 10755], "ttft_s": [50.583, 51.91, 50.672]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd4f093c724ebfd683feed823be35a11d4c009375 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T185415Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2

20261003T190629Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-stability

! fail stability T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
Started2026-10-04 05:06 AEST
Finished2026-10-04 05:16 AEST (10.4 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 30 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.854 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94fe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max9,732 tokens
context.headroom_min121,298 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls30 calls {"per_min": 3.0, "requests": 21}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps8.85 tok/s
window 1: 9.19 tok/swindow 2: 9.0 tok/swindow 3: 8.95 tok/swindow 4: 8.92 tok/swindow 5: 9.14 tok/smin 8.92 · max 9.19 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 274, "min_at_generation_s": 232.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps9.09 tok/s {"generation_seconds": 393.673, "generated_tokens": 3578.8}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 578.9, "requests": 21, "tool_calls": 30, "completion_tokens": 3601, "occupied_max": 9732}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 21.1, "requests": 1, "tool_calls": 1, "completion_tokens": 0, "occupied_max": 0}] list
tasks_over_64k[] instance ids
completion_tokens3,601 tokens {"finish_reasons": {"tool_calls": 21}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 30 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 8.85 tok/s, whole run 9.09 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd4f093c724ebfd683feed823be35a11d4c009375 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T190629Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2

20261003T191656Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
Started2026-10-04 05:16 AEST
Finished2026-10-04 05:31 AEST (15.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94fe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.50 tok/s
lab_prefill_tps199 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd4f093c724ebfd683feed823be35a11d4c009375 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T191656Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2

20261003T193228Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-load

✓ pass load T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
Started2026-10-04 05:32 AEST
Finished2026-10-04 05:33 AEST (0.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13a
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds51.9 s
host_ram_drop_gb115 GB
vram_ready_mib22,318 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit956afbaae18f13c44809e1d61a0a1568dca9db78 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T193228Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-load.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2

20261003T193321Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-lab

! fail lab T22b eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
Started2026-10-04 05:33 AEST
Finished2026-10-04 05:48 AEST (15.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13a
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps8.60 tok/s
lab_prefill_tps147 tok/s "context-gate prompt tokens / total request seconds"
mtp_acceptance_rate0.5403 fraction {"draft_tokens": 2889, "accepted_tokens": 1561, "source": "server log delta"}
mtp_accepted_tps8.60 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit956afbaae18f13c44809e1d61a0a1568dca9db78 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T193321Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2

20261003T194821Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-ttft

✓ pass TTFT T22b eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
Started2026-10-04 05:48 AEST
Finished2026-10-04 06:02 AEST (13.9 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13a
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps199 tok/s {"prompt_tokens": [10721, 10696, 10751], "ttft_s": [53.479, 53.836, 54.249]}
ttft_prefill_32k_tps191 tok/s {"prompt_tokens": [42813, 42832, 42764], "ttft_s": [223.74, 224.908, 224.251]}
ttft_prefill_tps199 tok/s {"prompt_tokens": [10721, 10696, 10751], "ttft_s": [53.479, 53.836, 54.249]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit956afbaae18f13c44809e1d61a0a1568dca9db78 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T194821Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-ttft.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2

20261003T200216Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-stability

✕ crash stability T22b eligibility: none failed stage: measure

Configik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
Started2026-10-04 06:02 AEST
Finished2026-10-04 06:12 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13a
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max1,641 tokens
context.headroom_min129,356 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls0 calls {"per_min": 0.0, "requests": 9}
parse_failures9 responses {"wire": {"unparsed_markup": 9}, "harness_format_errors": {"no_tool_call": 9}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors9 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tpsunavailable only 58.214 s of generation; need > 120 s for one window
decode_whole_run_tps12.5 tok/s {"generation_seconds": 58.214, "generated_tokens": 729.8}
tasks_attempted5 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomyes {"upstream_connection_failures": 54, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 41.0, "requests": 3, "tool_calls": 0, "completion_tokens": 274, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 36.8, "requests": 3, "tool_calls": 0, "completion_tokens": 248, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 34.6, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 341.4}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 146.2}] list
tasks_over_64k[] instance ids
completion_tokens739 tokens {"finish_reasons": {"stop": 9}}
mtp_accepted_tpsunavailable not produced (crash/measure)
mtp_acceptance_rateunavailable not produced (crash/measure)

Agentic summary: 600 s, 0 tool calls, 9 parse failures, 0 repetition hits, tasks solved 0/5, decode min window – tok/s, whole run 12.5 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-31
cpus_offlinenone
machine_checks_this_boot0
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit956afbaae18f13c44809e1d61a0a1568dca9db78 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T200216Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-stability.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012

20261003T220620Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-load

✓ pass load T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012
Started2026-10-04 08:06 AEST
Finished2026-10-04 08:07 AEST (0.7 min)
Exit statusNone
Notes–

Other runs of this config: load stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T220620Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit.json sha256 f7129eb274fa7a838d7a8928e9e72cec61e157259a237393b1f99b1945e722ab
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds39.9 s
host_ram_drop_gb3.35 GB
vram_ready_mib68,976 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2e517d99a59b71bbf630b44ce6204f3b85717adb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T220620Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-load.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012

20261003T221725Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-stability

! fail stability T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012
Started2026-10-04 08:17 AEST
Finished2026-10-04 08:19 AEST (1.9 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode floor not verifiable (too little generation); agent driver exited 1

Other runs of this config: load stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T220620Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit.json sha256 f7129eb274fa7a838d7a8928e9e72cec61e157259a237393b1f99b1945e722ab
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no successful request
context.headroom_minunavailable configured context unknown (set CONFIGURED_CTX)
duration_sunavailable driver produced no timing
tool_calls0 calls {"per_min": null, "requests": 0}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": null, "saved": "artifacts: proxy/repetition/"}
decode_window_min_tpsunavailable only 0.0 s of generation; need > 120 s for one window
decode_whole_run_tpsunavailable no generation recorded
tasks_attempted0 attempts {"unfinished_at_cap": 0, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solvedskipped no attempts
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[] list
tasks_over_64k[] instance ids
completion_tokens0 tokens {"finish_reasons": {}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: – s, 0 tool calls, 0 parse failures, 0 repetition hits, tasks solved –/0, decode min window – tok/s, whole run – tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd9944c1c5f811326cc98ee2701073db1981ab9b7 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261003T221725Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012

20261003T222524Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-load

✓ pass load T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
Started2026-10-04 08:25 AEST
Finished2026-10-04 08:27 AEST (1.7 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ✕ TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ce
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds101 s
host_ram_drop_gb40.4 GB
vram_ready_mib68,496 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitcf7ad504f20ebd9fc05fbafb689f4809d92ed210 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T222524Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012

20261003T222706Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
Started2026-10-04 08:27 AEST
Finished2026-10-04 08:51 AEST (24.6 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ✕ TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ce
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps12.3 tok/s
lab_prefill_tps158 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitcf7ad504f20ebd9fc05fbafb689f4809d92ed210 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T222706Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012

20261003T225145Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-ttft

✓ pass TTFT T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
Started2026-10-04 08:51 AEST
Finished2026-10-04 09:07 AEST (15.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ✕ TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ce
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps166 tok/s {"prompt_tokens": [10720, 10806, 10675], "ttft_s": [64.46, 64.694, 64.976]}
ttft_prefill_32k_tps175 tok/s {"prompt_tokens": [42882, 42861, 42885], "ttft_s": [244.773, 246.002, 245.262]}
ttft_prefill_tps166 tok/s {"prompt_tokens": [10720, 10806, 10675], "ttft_s": [64.46, 64.694, 64.976]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitcf7ad504f20ebd9fc05fbafb689f4809d92ed210 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T225145Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012

20261003T230715Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-stability

! fail stability T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
Started2026-10-04 09:07 AEST
Finished2026-10-04 09:17 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 26 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.388 < floor 15.0

Other runs of this config: load lab ! lab ✕ TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ce
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max8,118 tokens
context.headroom_min122,865 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls26 calls {"per_min": 2.6, "requests": 15}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps11.4 tok/s
window 1: 11.57 tok/swindow 2: 11.47 tok/swindow 3: 11.64 tok/smin 11.47 · max 11.64 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 143, "min_at_generation_s": 109.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps11.7 tok/s {"generation_seconds": 262.271, "generated_tokens": 3060.1}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 364.3, "requests": 12, "tool_calls": 20, "completion_tokens": 2740, "occupied_max": 8118}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 235.7, "requests": 4, "tool_calls": 6, "completion_tokens": 336, "occupied_max": 2784}] list
tasks_over_64k[] instance ids
completion_tokens3,076 tokens {"finish_reasons": {"tool_calls": 15}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 26 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 11.4 tok/s, whole run 11.7 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitcf7ad504f20ebd9fc05fbafb689f4809d92ed210 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T230715Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012

20261003T231744Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab

✕ crash lab T21 eligibility: none failed stage: harness

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
Started2026-10-04 09:17 AEST
Finished2026-10-04 09:29 AEST (11.3 min)
Exit status1
Notes–

Other runs of this config: load lab ! lab ✕ TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ce
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passedunavailable not produced (crash/harness)
lab_decode_c1_tpsunavailable not produced (crash/harness)
lab_prefill_tpsunavailable not produced (crash/harness)
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitcf7ad504f20ebd9fc05fbafb689f4809d92ed210 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T231744Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012

20261003T232915Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-load

✓ pass load T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
Started2026-10-04 09:29 AEST
Finished2026-10-04 09:30 AEST (1.3 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7af
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds79.7 s
host_ram_drop_gb2.99 GB
vram_ready_mib67,148 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd6382f80d3a94ab1a97311c8a27427f2b5c7f9cb clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T232915Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012

20261003T233035Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
Started2026-10-04 09:30 AEST
Finished2026-10-04 10:12 AEST (42.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7af
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps10.5 tok/s
lab_prefill_tps126 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit94e598f979af99cb8e62f9dacf2a0f1b258ccf3e clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261003T233035Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012

20261004T001236Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-ttft

✓ pass TTFT T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
Started2026-10-04 10:12 AEST
Finished2026-10-04 10:31 AEST (19.3 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7af
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps130 tok/s {"prompt_tokens": [10633, 10740, 10676], "ttft_s": [80.845, 98.056, 82.319]}
ttft_prefill_32k_tps143 tok/s {"prompt_tokens": [42717, 42962, 42901], "ttft_s": [298.369, 301.101, 295.768]}
ttft_prefill_tps130 tok/s {"prompt_tokens": [10633, 10740, 10676], "ttft_s": [80.845, 98.056, 82.319]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit376b9064b928086c6007ed16ab0d9267e111dd92 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T001236Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012

20261004T003153Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-stability

! fail stability T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
Started2026-10-04 10:31 AEST
Finished2026-10-04 10:42 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 22 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 9.855 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7af
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_max5,309 tokens
context.headroom_min125,652 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls22 calls {"per_min": 2.2, "requests": 16}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps9.86 tok/s
window 1: 10.15 tok/swindow 2: 10.0 tok/swindow 3: 10.01 tok/smin 10 · max 10.15 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 148, "min_at_generation_s": 195.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps10.1 tok/s {"generation_seconds": 267.587, "generated_tokens": 2697.7}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 225.2, "requests": 10, "tool_calls": 15, "completion_tokens": 1117, "occupied_max": 3970}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 374.8, "requests": 7, "tool_calls": 7, "completion_tokens": 1598, "occupied_max": 5309}] list
tasks_over_64k[] instance ids
completion_tokens2,715 tokens {"finish_reasons": {"tool_calls": 16}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 22 tool calls, 0 parse failures, 1 repetition hits, tasks solved 1/2, decode min window 9.86 tok/s, whole run 10.1 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit376b9064b928086c6007ed16ab0d9267e111dd92 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T003153Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012

20261004T004223Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
Started2026-10-04 10:42 AEST
Finished2026-10-04 11:09 AEST (27.2 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7af
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps10.4 tok/s
lab_prefill_tps132 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit376b9064b928086c6007ed16ab0d9267e111dd92 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T004223Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012

20261004T013855Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-load

✓ pass load T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012
Started2026-10-04 11:38 AEST
Finished2026-10-04 11:39 AEST (0.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T013855Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048.json sha256 d46ba43c1169b3fd2e5d00ace10dc78cf7122918e1c259debec506f8a0766d22
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds55.0 s
host_ram_drop_gb3.83 GB
vram_ready_mib65,124 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit640a2b5606870f5adba5d1a5f8a557a075bbdecc dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T013855Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012

20261004T013950Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012
Started2026-10-04 11:39 AEST
Finished2026-10-04 11:59 AEST (19.8 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T013855Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048.json sha256 d46ba43c1169b3fd2e5d00ace10dc78cf7122918e1c259debec506f8a0766d22
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps12.2 tok/s
lab_prefill_tps153 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit640a2b5606870f5adba5d1a5f8a557a075bbdecc dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T013950Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012

20261004T015940Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-ttft

✓ pass TTFT T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012
Started2026-10-04 11:59 AEST
Finished2026-10-04 12:15 AEST (16.0 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T013855Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048.json sha256 d46ba43c1169b3fd2e5d00ace10dc78cf7122918e1c259debec506f8a0766d22
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps159 tok/s {"prompt_tokens": [10694, 10665, 10720], "ttft_s": [67.037, 66.91, 67.278]}
ttft_prefill_32k_tps169 tok/s {"prompt_tokens": [42746, 42816, 42870], "ttft_s": [254.159, 253.337, 253.851]}
ttft_prefill_tps159 tok/s {"prompt_tokens": [10694, 10665, 10720], "ttft_s": [67.037, 66.91, 67.278]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit640a2b5606870f5adba5d1a5f8a557a075bbdecc dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T015940Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T021546Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-load

✓ pass load T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 12:15 AEST
Finished2026-10-04 12:16 AEST (0.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds54.9 s
host_ram_drop_gb4.03 GB
vram_ready_mib65,124 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd89b522703b1628ea955d1581e800faae3e8e5f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T021546Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T021642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 12:16 AEST
Finished2026-10-04 12:27 AEST (11.2 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps11.8 tok/s
lab_prefill_tps219 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd89b522703b1628ea955d1581e800faae3e8e5f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T021642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T022756Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-ttft

✓ pass TTFT T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 12:27 AEST
Finished2026-10-04 12:39 AEST (11.1 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps240 tok/s {"prompt_tokens": [10656, 10730, 10756], "ttft_s": [44.441, 45.48, 44.34]}
ttft_prefill_32k_tps242 tok/s {"prompt_tokens": [42911, 42857, 42784], "ttft_s": [177.67, 176.719, 177.974]}
ttft_prefill_tps240 tok/s {"prompt_tokens": [10656, 10730, 10756], "ttft_s": [44.441, 45.48, 44.34]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd89b522703b1628ea955d1581e800faae3e8e5f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T022756Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T023904Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-stability

! fail stability T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 12:39 AEST
Finished2026-10-04 12:49 AEST (10.4 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.035 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max6,174 tokens
context.headroom_min124,753 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls24 calls {"per_min": 2.4, "requests": 16}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps11.0 tok/s
window 1: 11.25 tok/swindow 2: 11.07 tok/smin 11.07 · max 11.25 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 78, "min_at_generation_s": 109.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps11.3 tok/s {"generation_seconds": 197.184, "generated_tokens": 2222.9}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 266.1, "requests": 13, "tool_calls": 19, "completion_tokens": 2034, "occupied_max": 6174}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 333.9, "requests": 4, "tool_calls": 5, "completion_tokens": 206, "occupied_max": 3300}] list
tasks_over_64k[] instance ids
completion_tokens2,240 tokens {"finish_reasons": {"tool_calls": 16}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 24 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 11.0 tok/s, whole run 11.3 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd89b522703b1628ea955d1581e800faae3e8e5f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T023904Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T024930Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 12:49 AEST
Finished2026-10-04 13:05 AEST (16.4 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps11.8 tok/s
lab_prefill_tps218 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd89b522703b1628ea955d1581e800faae3e8e5f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T024930Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2

20261004T030555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-load

✓ pass load T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2
Started2026-10-04 13:05 AEST
Finished2026-10-04 13:06 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T030555Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15.json sha256 72b9b0b1ecb00a8819e5a83e80deef181225276cbaa232ea9d7ed944c7391289
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds45.9 s
host_ram_drop_gb3.52 GB
vram_ready_mib21,944 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit0ace6784b9e48161d6b2ab80af2c1887ace48181 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T030555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2

20261004T030642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2
Started2026-10-04 13:06 AEST
Finished2026-10-04 13:29 AEST (22.3 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T030555Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15.json sha256 72b9b0b1ecb00a8819e5a83e80deef181225276cbaa232ea9d7ed944c7391289
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.50 tok/s
lab_prefill_tps201 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbc3d891210f23ffb6504b865b4216f82e08a42ef dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T030642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2

20261004T032900Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2
Started2026-10-04 13:29 AEST
Finished2026-10-04 13:41 AEST (12.2 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T030555Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15.json sha256 72b9b0b1ecb00a8819e5a83e80deef181225276cbaa232ea9d7ed944c7391289
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps213 tok/s {"prompt_tokens": [10652, 10738, 10639], "ttft_s": [50.031, 50.271, 51.458]}
ttft_prefill_32k_tps222 tok/s {"prompt_tokens": [42844, 42844, 42698], "ttft_s": [192.579, 195.095, 192.023]}
ttft_prefill_tps213 tok/s {"prompt_tokens": [10652, 10738, 10639], "ttft_s": [50.031, 50.271, 51.458]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbc3d891210f23ffb6504b865b4216f82e08a42ef dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T032900Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2

20261004T034114Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-load

✓ pass load T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
Started2026-10-04 13:41 AEST
Finished2026-10-04 13:42 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972be
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds45.9 s
host_ram_drop_gb4.55 GB
vram_ready_mib22,914 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2dbf31a09288732fa781855672b4d25178eb4682 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T034114Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2

20261004T034201Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
Started2026-10-04 13:42 AEST
Finished2026-10-04 13:55 AEST (13.9 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972be
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.60 tok/s
lab_prefill_tps288 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2dbf31a09288732fa781855672b4d25178eb4682 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T034201Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2

20261004T035555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
Started2026-10-04 13:55 AEST
Finished2026-10-04 14:04 AEST (8.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972be
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps321 tok/s {"prompt_tokens": [10699, 10766, 10755], "ttft_s": [33.318, 33.541, 34.552]}
ttft_prefill_32k_tps316 tok/s {"prompt_tokens": [42816, 42866, 42738], "ttft_s": [135.262, 135.596, 135.444]}
ttft_prefill_tps321 tok/s {"prompt_tokens": [10699, 10766, 10755], "ttft_s": [33.318, 33.541, 34.552]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2dbf31a09288732fa781855672b4d25178eb4682 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T035555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2

20261004T040423Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-stability

! fail stability T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
Started2026-10-04 14:04 AEST
Finished2026-10-04 14:14 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 27 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 8.847 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972be
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max6,632 tokens
context.headroom_min124,337 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls27 calls {"per_min": 2.7, "requests": 17}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps8.85 tok/s
window 1: 9.1 tok/swindow 2: 8.89 tok/swindow 3: 9.14 tok/swindow 4: 9.3 tok/swindow 5: 9.15 tok/swindow 6: 9.01 tok/smin 8.89 · max 9.3 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 314, "min_at_generation_s": 123.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps9.10 tok/s {"generation_seconds": 433.213, "generated_tokens": 3940.0}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 331.4, "requests": 11, "tool_calls": 18, "completion_tokens": 2064, "occupied_max": 6088}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 268.6, "requests": 7, "tool_calls": 9, "completion_tokens": 1894, "occupied_max": 6632}] list
tasks_over_64k[] instance ids
completion_tokens3,958 tokens {"finish_reasons": {"tool_calls": 17}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 27 tool calls, 0 parse failures, 1 repetition hits, tasks solved 1/2, decode min window 8.85 tok/s, whole run 9.10 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2dbf31a09288732fa781855672b4d25178eb4682 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T040423Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2

20261004T041451Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
Started2026-10-04 14:14 AEST
Finished2026-10-04 14:23 AEST (8.6 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972be
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.60 tok/s
lab_prefill_tps288 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2dbf31a09288732fa781855672b4d25178eb4682 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T041451Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012

20261004T042329Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012-load

✕ crash load T22c eligibility: none failed stage: load

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012
Started2026-10-04 14:23 AEST
Finished2026-10-04 14:24 AEST (0.7 min)
Exit status1
Notesche_init: CUDA2 KV buffer size = 570.81 MiB llama_init_from_model: KV self size = 748.00 MiB, c^KV (q8_0): 748.00 MiB, kv^T: not used llama_init_from_model: KV self indexer size = 704.00 MiB (f16) llama_init_from_model: CUDA_Host output buffer size = 0.59 MiB ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696 llama_init_from_model: failed to allocate compute buffers ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'

Launch

recorded by tools/run.py (artifacts/runs/20261004T042329Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210.json sha256 b1d4a0440963d486728825e444f7ac6736f7d39444e483cdf843d738cf41f923
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_secondsunavailable not produced (crash/load)
host_ram_drop_gbunavailable not produced (crash/load)
vram_ready_mibunavailable not produced (crash/load)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit562ad23a6a0d69a48c21a7611e7c2d9f0984a1ed dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T042329Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012-load.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012

20261004T042414Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012-load

✕ crash load T22c eligibility: none failed stage: load

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012
Started2026-10-04 14:24 AEST
Finished2026-10-04 14:25 AEST (0.8 min)
Exit status1
Notesche_init: CUDA2 KV buffer size = 570.81 MiB llama_init_from_model: KV self size = 816.00 MiB, c^KV (q8_0): 816.00 MiB, kv^T: not used llama_init_from_model: KV self indexer size = 768.00 MiB (f16) llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696 llama_init_from_model: failed to allocate compute buffers ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'

Launch

recorded by tools/run.py (artifacts/runs/20261004T042414Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210.json sha256 9f2b351bbcb5a9f58f9c86180a6023d2ce919c2149f9640d3778c925a0ec8881
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_secondsunavailable not produced (crash/load)
host_ram_drop_gbunavailable not produced (crash/load)
vram_ready_mibunavailable not produced (crash/load)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit0c585c1d628c0c4a04e8994c51e5954b718fabea dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T042414Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T042504Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-load

✓ pass load T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 14:25 AEST
Finished2026-10-04 14:26 AEST (1.5 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds91.2 s
host_ram_drop_gb3.05 GB
vram_ready_mib63,084 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T042504Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T042635Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 14:26 AEST
Finished2026-10-04 14:52 AEST (25.7 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps8.00 tok/s
lab_prefill_tps139 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T042635Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T045220Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-ttft

✓ pass TTFT T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 14:52 AEST
Finished2026-10-04 15:08 AEST (16.7 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps149 tok/s {"prompt_tokens": [10777, 10733, 10753], "ttft_s": [130.896, 71.944, 64.716]}
ttft_prefill_32k_tps176 tok/s {"prompt_tokens": [42666, 42838, 42755], "ttft_s": [243.017, 245.943, 242.684]}
ttft_prefill_tps149 tok/s {"prompt_tokens": [10777, 10733, 10753], "ttft_s": [130.896, 71.944, 64.716]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T045220Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T050859Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-stability

! fail stability T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 15:08 AEST
Finished2026-10-04 15:19 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.807 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max6,852 tokens
context.headroom_min123,939 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls25 calls {"per_min": 2.5, "requests": 19}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps7.81 tok/s
window 1: 7.91 tok/swindow 2: 7.96 tok/swindow 3: 8.06 tok/smin 7.91 · max 8.06 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 175, "min_at_generation_s": 107.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps7.95 tok/s {"generation_seconds": 294.515, "generated_tokens": 2340.5}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 277.5, "requests": 9, "tool_calls": 12, "completion_tokens": 1323, "occupied_max": 4597}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 322.5, "requests": 11, "tool_calls": 13, "completion_tokens": 1038, "occupied_max": 6852}] list
tasks_over_64k[] instance ids
completion_tokens2,361 tokens {"finish_reasons": {"tool_calls": 19}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 25 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 7.81 tok/s, whole run 7.95 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T050859Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-stability.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012

20261004T051928Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
Started2026-10-04 15:19 AEST
Finished2026-10-04 15:38 AEST (19.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps8.30 tok/s
lab_prefill_tps156 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T051928Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012

20261004T053844Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-load

✓ pass load T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Started2026-10-04 15:38 AEST
Finished2026-10-04 15:39 AEST (0.7 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds39.9 s
host_ram_drop_gb2.65 GB
vram_ready_mib65,510 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd8fc15971beec0df168ae2ee8c6c01aae2a7453 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T053844Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-load.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012

20261004T053925Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab

✓ pass lab T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Started2026-10-04 15:39 AEST
Finished2026-10-04 15:45 AEST (6.3 min)
Exit status0
Notesserved=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps56.9 tok/s
lab_prefill_tps605 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite17f05543581ac460774a08531a895566f2d209d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T053925Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012

20261004T054545Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-ttft

✓ pass TTFT T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Started2026-10-04 15:45 AEST
Finished2026-10-04 15:49 AEST (3.9 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps987 tok/s {"prompt_tokens": [13709, 13794, 13718], "ttft_s": [14.16, 13.87, 13.896]}
ttft_prefill_32k_tps852 tok/s {"prompt_tokens": [54701, 54603, 54830], "ttft_s": [64.015, 64.121, 64.836]}
ttft_prefill_tps987 tok/s {"prompt_tokens": [13709, 13794, 13718], "ttft_s": [14.16, 13.87, 13.896]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite17f05543581ac460774a08531a895566f2d209d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T054545Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-ttft.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012

20261004T054940Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-stability

! fail stability T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Started2026-10-04 15:49 AEST
Finished2026-10-04 16:00 AEST (10.4 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 70 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.626 < floor 50.0

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max39,226 tokens
context.headroom_min91,731 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls70 calls {"per_min": 7.0, "requests": 54}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps38.6 tok/s
window 1: 48.84 tok/swindow 2: 45.07 tok/swindow 3: 41.96 tok/swindow 4: 39.7 tok/smin 39.7 · max 48.84 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 209, "min_at_generation_s": 267.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps45.4 tok/s {"generation_seconds": 328.55, "generated_tokens": 14907.4}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 49.5, "requests": 11, "tool_calls": 16, "completion_tokens": 2006, "occupied_max": 5874}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 550.5, "requests": 44, "tool_calls": 54, "completion_tokens": 12968, "occupied_max": 39226}] list
tasks_over_64k[] instance ids
completion_tokens14,974 tokens {"finish_reasons": {"tool_calls": 54}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 70 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 38.6 tok/s, whole run 45.4 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite17f05543581ac460774a08531a895566f2d209d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T054940Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-stability.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012

20261004T060007Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab

✓ pass lab T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
Started2026-10-04 16:00 AEST
Finished2026-10-04 16:13 AEST (13.3 min)
Exit status0
Notesserved=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps57.1 tok/s
lab_prefill_tps608 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite17f05543581ac460774a08531a895566f2d209d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T060007Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2

20261004T061329Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-load

✓ pass load T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2
Started2026-10-04 16:13 AEST
Finished2026-10-04 16:14 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T061329Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192.json sha256 6a9c08474b62518d1e1c1d23f6b6b1f241e7a0e05c5faa5f176e3700eec9fefe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds48.9 s
host_ram_drop_gb5.84 GB
vram_ready_mib22,186 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit5af5149471babca3e64d893eeea35a3a1a43fe06 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T061329Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2

20261004T061418Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-lab

! fail lab T20b eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2
Started2026-10-04 16:14 AEST
Finished2026-10-04 16:23 AEST (9.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T061329Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192.json sha256 6a9c08474b62518d1e1c1d23f6b6b1f241e7a0e05c5faa5f176e3700eec9fefe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps9.10 tok/s
lab_prefill_tps370 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdd520bfd96d949a57962b56ed6cd8dd55f834916 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T061418Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2

20261004T062318Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-ttft

✓ pass TTFT T20b eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2
Started2026-10-04 16:23 AEST
Finished2026-10-04 16:30 AEST (6.7 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T061329Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192.json sha256 6a9c08474b62518d1e1c1d23f6b6b1f241e7a0e05c5faa5f176e3700eec9fefe
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192
envCUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps384 tok/s {"prompt_tokens": [10746, 10734, 10753], "ttft_s": [27.965, 27.98, 27.965]}
ttft_prefill_32k_tps404 tok/s {"prompt_tokens": [42688, 42881, 42819], "ttft_s": [105.798, 105.946, 106.334]}
ttft_prefill_tps384 tok/s {"prompt_tokens": [10746, 10734, 10753], "ttft_s": [27.965, 27.98, 27.965]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdd520bfd96d949a57962b56ed6cd8dd55f834916 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T062318Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-ttft.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012

20261004T063003Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-load

✓ pass load T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012
Started2026-10-04 16:30 AEST
Finished2026-10-04 16:30 AEST (0.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T063003Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210.json sha256 e0b729222e80461545b33c6e10b8ff1d8ecaaf2fde1596824ce2a3e98b0ce76c
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds54.9 s
host_ram_drop_gb4.93 GB
vram_ready_mib65,688 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9a178e2fe77cc647e9542e1742a52c38931f07e1 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T063003Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-load.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012

20261004T063058Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-lab

! fail lab T21 eligibility: none failed stage: gates

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012
Started2026-10-04 16:30 AEST
Finished2026-10-04 16:48 AEST (17.9 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T063003Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210.json sha256 e0b729222e80461545b33c6e10b8ff1d8ecaaf2fde1596824ce2a3e98b0ce76c
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps11.2 tok/s
lab_prefill_tps258 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9a178e2fe77cc647e9542e1742a52c38931f07e1 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T063058Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-lab.json

Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012

20261004T064850Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-ttft

✓ pass TTFT T21 eligibility: none

Configllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012
Started2026-10-04 16:48 AEST
Finished2026-10-04 16:58 AEST (9.3 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T063003Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g/docker-run.txt)
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210.json sha256 e0b729222e80461545b33c6e10b8ff1d8ecaaf2fde1596824ce2a3e98b0ce76c
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps298 tok/s {"prompt_tokens": [10829, 10677, 10700], "ttft_s": [37.303, 35.77, 35.957]}
ttft_prefill_32k_tps288 tok/s {"prompt_tokens": [42788, 42675, 42983], "ttft_s": [148.453, 148.434, 149.751]}
ttft_prefill_tps298 tok/s {"prompt_tokens": [10829, 10677, 10700], "ttft_s": [37.303, 35.77, 35.957]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9a178e2fe77cc647e9542e1742a52c38931f07e1 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T064850Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-ttft.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012

20261004T065809Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-load

✓ pass load T22c eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
Started2026-10-04 16:58 AEST
Finished2026-10-04 16:58 AEST (0.7 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds44.3 s
host_ram_drop_gb65.5 GB
vram_ready_mib68,156 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T065809Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-load.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012

20261004T065853Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab

! fail lab T22c eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
Started2026-10-04 16:58 AEST
Finished2026-10-04 17:10 AEST (12.0 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps14.0 tok/s
lab_prefill_tps185 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T065853Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012

20261004T071053Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-ttft

✓ pass TTFT T22c eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
Started2026-10-04 17:10 AEST
Finished2026-10-04 17:21 AEST (10.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps283 tok/s {"prompt_tokens": [10728, 10711, 10734], "ttft_s": [37.814, 37.788, 37.935]}
ttft_prefill_32k_tps253 tok/s {"prompt_tokens": [42827, 42804, 42935], "ttft_s": [167.744, 169.791, 169.931]}
ttft_prefill_tps283 tok/s {"prompt_tokens": [10728, 10711, 10734], "ttft_s": [37.814, 37.788, 37.935]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T071053Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-ttft.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012

20261004T072115Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-stability

! fail stability T22c eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
Started2026-10-04 17:21 AEST
Finished2026-10-04 17:31 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 68 tool-call parse failure(s); decode window min 13.255 < floor 15.0

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max1,874 tokens
context.headroom_min129,132 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls0 calls {"per_min": 0.0, "requests": 68}
parse_failures68 responses {"wire": {"unparsed_markup": 68}, "harness_format_errors": {"no_tool_call": 68}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors68 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps13.3 tok/s
window 1: 13.31 tok/swindow 2: 13.29 tok/swindow 3: 13.29 tok/swindow 4: 13.35 tok/swindow 5: 13.34 tok/smin 13.29 · max 13.35 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 299, "min_at_generation_s": 112.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps13.3 tok/s {"generation_seconds": 418.663, "generated_tokens": 5576.2}
tasks_attempted23 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 30.9, "requests": 3, "tool_calls": 0, "completion_tokens": 246, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 27.9, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 37.7, "requests": 3, "tool_calls": 0, "completion_tokens": 348, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 31.6, "requests": 3, "tool_calls": 0, "completion_tokens": 263, "occupied_max": 1874}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 31.2, "requests": 3, "tool_calls": 0, "completion_tokens": 260, "occupied_max": 1770}, {"attempt": 6, "round": 1, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.1, "requests": 3, "tool_calls": 0, "completion_tokens": 181, "occupied_max": 1716}, {"attempt": 7, "round": 1, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 27.6, "requests": 3, "tool_calls": 0, "completion_tokens": 214, "occupied_max": 1641}, {"attempt": 8, "round": 1, "instance_id": "sympy__sympy-20590", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 39.7, "requests": 3, "tool_calls": 0, "completion_tokens": 375, "occupied_max": 1578}, {"attempt": 9, "round": 2, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 23.2, "requests": 3, "tool_calls": 0, "completion_tokens": 240, "occupied_max": 1559}, {"attempt": 10, "round": 2, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 21.1, "requests": 3, "tool_calls": 0, "completion_tokens": 212, "occupied_max": 1641}, {"attempt": 11, "round": 2, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 26.7, "requests": 3, "tool_calls": 0, "completion_tokens": 286, "occupied_max": 1609}, {"attempt": 12, "round": 2, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.0, "requests": 3, "tool_calls": 0, "completion_tokens": 262, "occupied_max": 1874}, {"attempt": 13, "round": 2, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 24.6, "requests": 3, "tool_calls": 0, "completion_tokens": 257, "occupied_max": 1770}, {"attempt": 14, "round": 2, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 19.0, "requests": 3, "tool_calls": 0, "completion_tokens": 184, "occupied_max": 1716}, {"attempt": 15, "round": 2, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 20.8, "requests": 3, "tool_calls": 0, "completion_tokens": 208, "occupied_max": 1641}, {"attempt": 16, "round": 2, "instance_id": "sympy__sympy-20590", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 29.2, "requests": 3, "tool_calls": 0, "completion_tokens": 319, "occupied_max": 1578}, {"attempt": 17, "round": 3, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 23.2, "requests": 3, "tool_calls": 0, "completion_tokens": 240, "occupied_max": 1559}, {"attempt": 18, "round": 3, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 21.1, "requests": 3, "tool_calls": 0, "completion_tokens": 212, "occupied_max": 1641}, {"attempt": 19, "round": 3, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 26.8, "requests": 3, "tool_calls": 0, "completion_tokens": 286, "occupied_max": 1609}, {"attempt": 20, "round": 3, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.0, "requests": 3, "tool_calls": 0, "completion_tokens": 262, "occupied_max": 1874}, {"attempt": 21, "round": 3, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 24.6, "requests": 3, "tool_calls": 0, "completion_tokens": 257, "occupied_max": 1770}, {"attempt": 22, "round": 3, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 19.0, "requests": 3, "tool_calls": 0, "completion_tokens": 184, "occupied_max": 1716}, {"attempt": 23, "round": 3, "instance_id": "pytest-dev__pytest-7373", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 19.0, "requests": 3, "tool_calls": 0, "completion_tokens": 133, "occupied_max": 1522}] list
tasks_over_64k[] instance ids
completion_tokens5,646 tokens {"finish_reasons": {"stop": 68}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 0 tool calls, 68 parse failures, 0 repetition hits, tasks solved 0/23, decode min window 13.3 tok/s, whole run 13.3 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T072115Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-stability.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012

20261004T073119Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab

! fail lab T22c eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
Started2026-10-04 17:31 AEST
Finished2026-10-04 17:42 AEST (11.2 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! lab ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341
MTP / speculativeoff
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps13.4 tok/s
lab_prefill_tps189 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T073119Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012

20261004T074233Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012-load

✕ crash load T22c eligibility: none failed stage: load

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012
Started2026-10-04 17:42 AEST
Finished2026-10-04 17:43 AEST (0.9 min)
Exit status1
Notes c^KV (q8_0): 68.00 MiB, kv^T: not used llama_init_from_model: KV self indexer size = 64.00 MiB (f16) llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2754.25 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 2888039424 llama_init_from_model: failed to allocate compute buffers ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs common_speculative_init: failed to create MTP context srv init: failed to initialize recurrent speculative context ~ggml_backend_cuda_context: have 13 graphs ~ggml_backend_cuda_context: have 14 graphs ~ggml_backend_cuda_context: have 11 graphs

Launch

recorded by tools/run.py (artifacts/runs/20261004T074233Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 4096
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096.json sha256 d21078e38ec9e96da581e9003a2aaae041591909413dbf70738357df0829341e
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_secondsunavailable not produced (crash/load)
host_ram_drop_gbunavailable not produced (crash/load)
vram_ready_mibunavailable not produced (crash/load)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitf3cbbe1b98ba109e9c4abaac8d34bc7b6e2775ae dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T074233Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012-load.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012

20261004T080048Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-load

✓ pass load T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012
Started2026-10-04 18:00 AEST
Finished2026-10-04 18:01 AEST (0.7 min)
Exit statusNone
Notes–

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T080048Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048.json sha256 e882737bdeeadb764f93582d803f25626630b8b0a09fb20716d71d1c2c7be516
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds39.9 s
host_ram_drop_gb3.74 GB
vram_ready_mib65,396 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit65a204856ba6e28ca395fa6aab47a88e41c62a1a dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T080048Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-load.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012

20261004T080128Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-lab

✓ pass lab T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012
Started2026-10-04 18:01 AEST
Finished2026-10-04 18:06 AEST (5.5 min)
Exit status0
Notesserved=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T080048Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048.json sha256 e882737bdeeadb764f93582d803f25626630b8b0a09fb20716d71d1c2c7be516
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps44.3 tok/s
lab_prefill_tps717 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit65a204856ba6e28ca395fa6aab47a88e41c62a1a dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T080128Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-lab.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012

20261004T080657Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-ttft

✓ pass TTFT T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012
Started2026-10-04 18:06 AEST
Finished2026-10-04 18:10 AEST (3.6 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T080048Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048.json sha256 e882737bdeeadb764f93582d803f25626630b8b0a09fb20716d71d1c2c7be516
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps1,067 tok/s {"prompt_tokens": [13695, 13734, 13847], "ttft_s": [12.963, 12.872, 12.965]}
ttft_prefill_32k_tps931 tok/s {"prompt_tokens": [54728, 54685, 54604], "ttft_s": [58.434, 58.71, 58.906]}
ttft_prefill_tps1,067 tok/s {"prompt_tokens": [13695, 13734, 13847], "ttft_s": [12.963, 12.872, 12.965]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit65a204856ba6e28ca395fa6aab47a88e41c62a1a dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T080657Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-ttft.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012

20261004T081034Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-load

✓ pass load T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012
Started2026-10-04 18:10 AEST
Finished2026-10-04 18:11 AEST (0.7 min)
Exit statusNone
Notes–

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T081034Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096.json sha256 bb5c0e11a6559307e48f7f8f285d3439f9c7872614fa95ee033b2d48ca370dbd
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds39.9 s
host_ram_drop_gb4.34 GB
vram_ready_mib65,508 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbfcf5657ea4ed56f452dc532e3311cf77750c3d9 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T081034Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-load.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012

20261004T081114Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-lab

✓ pass lab T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012
Started2026-10-04 18:11 AEST
Finished2026-10-04 18:19 AEST (8.1 min)
Exit status0
Notesserved=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T081034Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096.json sha256 bb5c0e11a6559307e48f7f8f285d3439f9c7872614fa95ee033b2d48ca370dbd
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps35.0 tok/s
lab_prefill_tps713 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbfcf5657ea4ed56f452dc532e3311cf77750c3d9 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T081114Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-lab.json

Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012

20261004T081919Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-ttft

✓ pass TTFT T11 eligibility: none

Configllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012
Started2026-10-04 18:19 AEST
Finished2026-10-04 18:23 AEST (3.8 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261004T081034Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt)
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096
Imageghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Digestsha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5
Enginellama.cpp
Engine source–
Launch filelaunches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096.json sha256 bb5c0e11a6559307e48f7f8f285d3439f9c7872614fa95ee033b2d48ca370dbd
MTP / speculativeoff
argv/app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps980 tok/s {"prompt_tokens": [13680, 13707, 13810], "ttft_s": [14.13, 13.965, 14.096]}
ttft_prefill_32k_tps881 tok/s {"prompt_tokens": [54557, 54627, 54737], "ttft_s": [61.779, 62.069, 62.105]}
ttft_prefill_tps980 tok/s {"prompt_tokens": [13680, 13707, 13810], "ttft_s": [14.13, 13.965, 14.096]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbfcf5657ea4ed56f452dc532e3311cf77750c3d9 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T081919Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-ttft.json

Runs / none

20261004T091142Z-hw-bw-gpu1-current-hw_bw

✓ pass hw bw T05a eligibility: none

Confighw-bw-gpu1-current
Started2026-10-04 19:11 AEST
Finished2026-10-04 19:11 AEST (0.2 min)
Exit status0
Notes1 GiB pinned transfers; full table in detail

Launch

no container: static measurement

Metrics

context.configured–
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
h2d_gbps13.5 GB/s {"1MB_pin": 13.14, "1MB_page": 13.01, "4MB_pin": 13.39, "4MB_page": 13.12, "16MB_pin": 13.45, "16MB_page": 13.18, "64MB_pin": 13.47, "64MB_page": 13.18, "256MB_pin": 13.47, "256MB_page": 13.07, "1024MB_pin": 13.47, "1024MB_page": 13.12}
d2h_gbps13.2 GB/s {"1MB_pin": 12.95, "1MB_page": 6.3, "4MB_pin": 13.14, "4MB_page": 10.05, "16MB_pin": 13.2, "16MB_page": 11.16, "64MB_pin": 13.21, "64MB_page": 11.79, "256MB_pin": 13.21, "256MB_page": 12.04, "1024MB_pin": 13.22, "1024MB_page": 12.05}
host_memcpy_gbps16.6 GB/s

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit00709a719d4c6057fbeb8f52eadd20c2bbe2f46d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/none/20261004T091142Z-hw-bw-gpu1-current-hw_bw.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012

20261004T091154Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-load

✓ pass load T22c eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
Started2026-10-04 19:11 AEST
Finished2026-10-04 19:12 AEST (0.8 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06c
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds48.6 s
host_ram_drop_gb83.3 GB
vram_ready_mib64,680 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitc36c7e58741670e5e869f6857478b1e198932e9a dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T091154Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-load.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012

20261004T091243Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-lab

! fail lab T22c eligibility: none failed stage: gates

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
Started2026-10-04 19:12 AEST
Finished2026-10-04 19:26 AEST (13.5 min)
Exit status0
Notesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06c
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}
lab_decode_c1_tps10.2 tok/s
lab_prefill_tps166 tok/s "context-gate prompt tokens / total request seconds"
mtp_acceptance_rate0.5137 fraction {"draft_tokens": 3366, "accepted_tokens": 1729, "source": "server log delta"}
mtp_accepted_tps10.2 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit151350a4d65da543200768e08741f6eb5409c3ed dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T091243Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-lab.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012

20261004T092615Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-ttft

✓ pass TTFT T22c eligibility: none

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
Started2026-10-04 19:26 AEST
Finished2026-10-04 19:38 AEST (11.7 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06c
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps248 tok/s {"prompt_tokens": [10798, 10674, 10665], "ttft_s": [43.785, 43.033, 42.927]}
ttft_prefill_32k_tps224 tok/s {"prompt_tokens": [42970, 42867, 42822], "ttft_s": [191.654, 191.974, 191.365]}
ttft_prefill_tps248 tok/s {"prompt_tokens": [10798, 10674, 10665], "ttft_s": [43.785, 43.033, 42.927]}

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit42d955c4fa51a99398f09218c4a4c133bcf18938 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T092615Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-ttft.json

Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012

20261004T093800Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-stability

✕ crash stability T22c eligibility: none failed stage: measure

Configik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
Started2026-10-04 19:38 AEST
Finished2026-10-04 19:48 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)

Other runs of this config: load lab ! TTFT stability ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt)
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
Imagelocal-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Digestsha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1
Engineik_llama.cpp
Engine sourcehttps://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678
Launch filelaunches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06c
MTP / speculative{"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}
argv/app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168
envCUDA_DEVICE_ORDER=PCI_BUS_ID
CUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json

Metrics

context.configured131,072 tokens
context.occupied_max1,641 tokens
context.headroom_min129,336 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls0 calls {"per_min": 0.0, "requests": 9}
parse_failures9 responses {"wire": {"unparsed_markup": 9}, "harness_format_errors": {"no_tool_call": 9}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors9 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tpsunavailable only 41.172 s of generation; need > 120 s for one window
decode_whole_run_tps15.6 tok/s {"generation_seconds": 41.172, "generated_tokens": 642.7}
tasks_attempted5 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomyes {"upstream_connection_failures": 54, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 26.8, "requests": 3, "tool_calls": 0, "completion_tokens": 208, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 28.3, "requests": 3, "tool_calls": 0, "completion_tokens": 227, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 27.1, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 343.6}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 174.2}] list
tasks_over_64k[] instance ids
completion_tokens652 tokens {"finish_reasons": {"stop": 9}}
mtp_accepted_tpsunavailable not produced (crash/measure)
mtp_acceptance_rateunavailable not produced (crash/measure)

Agentic summary: 600 s, 0 tool calls, 9 parse failures, 0 repetition hits, tasks solved 0/5, decode min window – tok/s, whole run 15.6 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 3 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0●NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1●NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit42d955c4fa51a99398f09218c4a4c133bcf18938 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T093800Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-stability.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T101015Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

✓ pass load T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:10 AEST
Finished2026-10-04 20:14 AEST (3.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds236 s
host_ram_drop_gb63.5 GB
vram_ready_mib20,818 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbdd5d466ac35145adc76b9f18538d7ef1373acce dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T101015Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T101411Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

✓ pass lab T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:14 AEST
Finished2026-10-04 20:19 AEST (5.4 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.6 tok/s
lab_prefill_tps2,433 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbdd5d466ac35145adc76b9f18538d7ef1373acce dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T101411Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T101937Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

✓ pass kit sweep T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:19 AEST
Finished2026-10-04 20:34 AEST (15.2 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,289 tok/s {"min": 2269.6, "max": 2294.4, "n": 3}
prefill_16k_tps2,240 tok/s {"min": 2233.0, "max": 2247.3, "n": 3}
prefill_32k_tps2,678 tok/s {"min": 2672.6, "max": 2681.2, "n": 3}
prefill_64k_tps2,653 tok/s {"min": 2645.1, "max": 2655.6, "n": 3}
decode_c1_tps48.2 tok/s {"min": 47.55, "max": 48.85, "power_w": null}
decode_c2_tps53.1 tok/s {"min": 51.57, "max": 54.71, "power_w": null}
decode_c3_tps55.9 tok/s {"min": 55.26, "max": 56.61, "power_w": null}
decode_c4_tps53.3 tok/s {"min": 51.51, "max": 55.06, "power_w": null}
decode_c1_32k_tps49.9 tok/s {"min": 49.58, "max": 50.2, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T101937Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T103452Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

✓ pass TTFT T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:34 AEST
Finished2026-10-04 20:36 AEST (1.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,017 tok/s {"prompt_tokens": [13801, 13745, 13788], "ttft_s": [6.842, 6.83, 6.822]}
ttft_prefill_32k_tps2,687 tok/s {"prompt_tokens": [54693, 54557, 54874], "ttft_s": [20.353, 20.312, 20.371]}
ttft_prefill_tps2,017 tok/s {"prompt_tokens": [13801, 13745, 13788], "ttft_s": [6.842, 6.83, 6.822]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T103452Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T103614Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

✓ pass quality panel T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:36 AEST
Finished2026-10-04 20:36 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.98867 fraction
mean_kl0.00096647 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T103614Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T103641Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

! fail stability T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:36 AEST
Finished2026-10-04 20:47 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 67 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.939 < floor 50.0

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max36,327 tokens
context.headroom_min168,259 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls67 calls {"per_min": 6.7, "requests": 46}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps37.9 tok/s
window 1: 39.97 tok/swindow 2: 43.13 tok/swindow 3: 42.36 tok/swindow 4: 46.42 tok/swindow 5: 43.39 tok/swindow 6: 42.38 tok/smin 39.97 · max 46.42 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 313, "min_at_generation_s": 87.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps44.1 tok/s {"generation_seconds": 432.859, "generated_tokens": 19093.9}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 216.5, "requests": 17, "tool_calls": 29, "completion_tokens": 6657, "occupied_max": 13921}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 383.5, "requests": 30, "tool_calls": 38, "completion_tokens": 12541, "occupied_max": 36327}] list
tasks_over_64k[] instance ids
completion_tokens19,198 tokens {"finish_reasons": {"tool_calls": 46}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 67 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 37.9 tok/s, whole run 44.1 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T103641Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2

20261004T104713Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72

✓ pass lab T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
Started2026-10-04 20:47 AEST
Finished2026-10-04 20:52 AEST (5.3 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Digestsha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps54.5 tok/s
lab_prefill_tps2,596 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T104713Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2

20261004T105250Z-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2-load

✕ crash load T15 eligibility: none failed stage: load

Configsglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2
Started2026-10-04 20:52 AEST
Finished2026-10-04 20:55 AEST (3.0 min)
Exit status0
Notes_glue/offload_moe_method.py", line 86, in get_runtime store = HostExpertStore(model_dir, layers, num_experts, hidden, inter, bits, prefix=prefix) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/trellis-serve/cuda/src/sglang_exl3/kernels/offload_store.py", line 62, in __init__ raise NotImplementedError("HostExpertStore: fast relayout implemented for K=3 experts only") NotImplementedError: HostExpertStore: fast relayout implemented for K=3 experts only [2026-10-04 10:55:46] Received sigquit from a child process. It usually means the child failed. [2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=True, pid=1 [2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=False, pid=1

Launch

recorded by tools/run.py (artifacts/runs/20261004T105250Z-lab-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4:/models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb.json sha256 0d170921f6e46f05cad846a1404d158b6e85e4fffe18eeec9a2d7d70ec79a969
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@65c895314393431c09050b2e04e250836b3a6eb4 · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_secondsunavailable not produced (crash/load)
host_ram_drop_gbunavailable not produced (crash/load)
vram_ready_mibunavailable not produced (crash/load)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdf0d10f31ae46a27f371fda8e1f15e40dfa878f0 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T105250Z-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2

20261004T105602Z-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2-load

✕ crash load T15 eligibility: none failed stage: load

Configsglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2
Started2026-10-04 20:56 AEST
Finished2026-10-04 20:56 AEST (0.9 min)
Exit status0
Notes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 116, in __init__ t = parse_table(model_path) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 94, in parse_table raise ValueError(f"{path}: shard_N.trellis tensors missing or not contiguous") ValueError: /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6/ngram_embedding.safetensors: shard_N.trellis tensors missing or not contiguous [2026-10-04 10:56:54] Received sigquit from a child process. It usually means the child failed. [2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=True, pid=1 [2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=False, pid=1

Launch

recorded by tools/run.py (artifacts/runs/20261004T105602Z-lab-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6:/models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Digestsha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb.json sha256 ce41d62d49f95e441684dbb7afcbfd2cd54c0cdf49476dda70cda1eca5bc7928
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@55a732e0c4c3d4614bc42b68493bb930d9b02c0a · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_secondsunavailable not produced (crash/load)
host_ram_drop_gbunavailable not produced (crash/load)
vram_ready_mibunavailable not produced (crash/load)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbd62e89c96f97db97945fd89be01017a19fea7c3 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T105602Z-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2-load.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T105700Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load

✓ pass load T23 eligibility: none

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 20:57 AEST
Finished2026-10-04 21:02 AEST (5.0 min)
Exit statusNone
Notes–

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds300 s
host_ram_drop_gb120 GB
vram_ready_mib23,322 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T105700Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T110201Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab

! fail lab T23 eligibility: none failed stage: gates

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 21:02 AEST
Finished2026-10-04 21:12 AEST (11.0 min)
Exit status0
Notesserved=glm-5.3-flash

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps7.40 tok/s
lab_prefill_tps739 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T110201Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T111300Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft

! fail TTFT T23 eligibility: none failed stage: invalid measurement

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 21:13 AEST
Finished2026-10-04 21:17 AEST (4.2 min)
Exit status0
NotesINVALIDATED: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2. headline length 8192; prompts carry random nonces

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tpsunavailable invalidated: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2
ttft_prefill_32k_tpsunavailable invalidated: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2
ttft_prefill_tpsunavailable invalidated: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T111300Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T111712Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-stability

! fail stability T23 eligibility: none

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 21:17 AEST
Finished2026-10-04 21:27 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 21 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 3.356 < floor 15.0

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_max4,589 tokens
context.headroom_min126,405 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls21 calls {"per_min": 2.1, "requests": 14}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps3.36 tok/s
window 1: 6.08 tok/swindow 2: 5.96 tok/smin 5.96 · max 6.08 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 99, "min_at_generation_s": 101.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps6.56 tok/s {"generation_seconds": 218.025, "generated_tokens": 1431.1}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 385.2, "requests": 11, "tool_calls": 16, "completion_tokens": 1605, "occupied_max": 4589}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 214.8, "requests": 4, "tool_calls": 5, "completion_tokens": 179, "occupied_max": 2222}] list
tasks_over_64k[] instance ids
completion_tokens1,784 tokens {"finish_reasons": {"tool_calls": 14}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 21 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 3.36 tok/s, whole run 6.56 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T111712Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-stability.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T112744Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab

! fail lab T23 eligibility: none failed stage: gates

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 21:27 AEST
Finished2026-10-04 22:05 AEST (37.9 min)
Exit status0
Notesserved=glm-5.3-flash

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps7.50 tok/s
lab_prefill_tps720 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T112744Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T120556Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:05 AEST
Finished2026-10-04 22:10 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds260 s
host_ram_drop_gb65.9 GB
vram_ready_mib20,822 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T120556Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T121018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:10 AEST
Finished2026-10-04 22:15 AEST (4.7 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.8 tok/s
lab_prefill_tps2,419 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T121018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T121501Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:15 AEST
Finished2026-10-04 22:30 AEST (15.7 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,281 tok/s {"min": 2261.8, "max": 2289.8, "n": 3}
prefill_16k_tps2,237 tok/s {"min": 2234.0, "max": 2244.7, "n": 3}
prefill_32k_tps2,678 tok/s {"min": 2670.7, "max": 2679.3, "n": 3}
prefill_64k_tps2,649 tok/s {"min": 2643.2, "max": 2654.6, "n": 3}
decode_c1_tps47.4 tok/s {"min": 46.48, "max": 48.27, "power_w": null}
decode_c2_tps53.1 tok/s {"min": 51.34, "max": 54.86, "power_w": null}
decode_c3_tps60.6 tok/s {"min": 57.47, "max": 63.7, "power_w": null}
decode_c4_tps53.7 tok/s {"min": 52.62, "max": 54.77, "power_w": null}
decode_c1_32k_tps48.5 tok/s {"min": 48.52, "max": 48.57, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T121501Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T123041Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:30 AEST
Finished2026-10-04 22:32 AEST (1.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,009 tok/s {"prompt_tokens": [13741, 13755, 13738], "ttft_s": [6.864, 6.84, 6.838]}
ttft_prefill_32k_tps2,681 tok/s {"prompt_tokens": [54658, 54711, 54684], "ttft_s": [20.392, 20.395, 20.396]}
ttft_prefill_tps2,009 tok/s {"prompt_tokens": [13741, 13755, 13738], "ttft_s": [6.864, 6.84, 6.838]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T123041Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T123204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality panel T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:32 AEST
Finished2026-10-04 22:32 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.98967 fraction
mean_kl0.00097327 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T123204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T123230Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✕ crash stability T13 eligibility: none failed stage: measure

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:32 AEST
Finished2026-10-04 22:42 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation)

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no successful request
context.headroom_minunavailable configured context unknown (set CONFIGURED_CTX)
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls0 calls {"per_min": 0.0, "requests": 0}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": null, "saved": "artifacts: proxy/repetition/"}
decode_window_min_tpsunavailable only 0.0 s of generation; need > 120 s for one window
decode_whole_run_tpsunavailable no generation recorded
tasks_attempted3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomyes {"upstream_connection_failures": 77, "upstream_http_errors": 1, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 267.3}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 261.8}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 71.0}] list
tasks_over_64k[] instance ids
completion_tokens0 tokens {"finish_reasons": {}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 0 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/3, decode min window – tok/s, whole run – tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T123230Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2

20261004T124252Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2
Started2026-10-04 22:42 AEST
Finished2026-10-04 22:47 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load lab ✕ TTFT ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T124252Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6.json sha256 90d5526c99bf2c3b87f25e51276eed60130a38e3a34bc2991695048a8bb33eb9
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds257 s
host_ram_drop_gb65.3 GB
vram_ready_mib21,706 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdb208909a5ae27d314ff09d53d7cf803d67fda9c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T124252Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2

20261004T124710Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✕ crash lab T14a eligibility: none failed stage: harness

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2
Started2026-10-04 22:47 AEST
Finished2026-10-04 22:48 AEST (1.7 min)
Exit status1
Notes–

Other runs of this config: load lab ✕ TTFT ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T124252Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6.json sha256 90d5526c99bf2c3b87f25e51276eed60130a38e3a34bc2991695048a8bb33eb9
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passedunavailable not produced (crash/harness)
lab_decode_c1_tpsunavailable not produced (crash/harness)
lab_prefill_tpsunavailable not produced (crash/harness)
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdb208909a5ae27d314ff09d53d7cf803d67fda9c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T124710Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2

20261004T124852Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✕ crash TTFT T14a eligibility: none failed stage: measure

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2
Started2026-10-04 22:48 AEST
Finished2026-10-04 22:49 AEST (0.3 min)
Exit status1
Notes–

Other runs of this config: load lab ✕ TTFT ✕

Launch

recorded by tools/run.py (artifacts/runs/20261004T124252Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6.json sha256 90d5526c99bf2c3b87f25e51276eed60130a38e3a34bc2991695048a8bb33eb9
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_tpsunavailable not produced (crash/measure)

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdb208909a5ae27d314ff09d53d7cf803d67fda9c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T124852Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T124925Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load

✓ pass load T23 eligibility: none

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 22:49 AEST
Finished2026-10-04 22:54 AEST (4.6 min)
Exit statusNone
Notes–

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T124925Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds276 s
host_ram_drop_gb121 GB
vram_ready_mib23,322 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite2424ec92249689799954b79c057d915d7f402f1 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T124925Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load.json

Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2

20261004T125401Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft

✓ pass TTFT T23 eligibility: none

Configexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
Started2026-10-04 22:54 AEST
Finished2026-10-04 22:58 AEST (4.2 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T124925Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt)
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
Imageghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Digestsha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa
Launch filelaunches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17fa
MTP / speculativeoff
argv/opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash
envGLM53_MODE=exact
GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw
GLM53_MODEL_DOWNLOAD=0
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps483 tok/s {"prompt_tokens": [10753, 10793, 10717], "ttft_s": [19.173, 22.332, 25.068]}
ttft_prefill_32k_tps688 tok/s {"prompt_tokens": [42865, 42692, 42811], "ttft_s": [62.271, 62.501, 60.752]}
ttft_prefill_tps483 tok/s {"prompt_tokens": [10753, 10793, 10717], "ttft_s": [19.173, 22.332, 25.068]}

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite2424ec92249689799954b79c057d915d7f402f1 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261004T125401Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T125827Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 22:58 AEST
Finished2026-10-04 23:02 AEST (4.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T125827Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds264 s
host_ram_drop_gb65.7 GB
vram_ready_mib20,844 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9229192eace16fc1782f479846e0194aea593ba9 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T125827Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T130251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 23:02 AEST
Finished2026-10-04 23:13 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.676 < floor 50.0

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T125827Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max25,309 tokens
context.headroom_min178,942 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls71 calls {"per_min": 7.1, "requests": 47}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps39.7 tok/s
window 1: 44.31 tok/swindow 2: 43.52 tok/swindow 3: 43.73 tok/swindow 4: 44.87 tok/swindow 5: 41.92 tok/smin 41.92 · max 44.87 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 254, "min_at_generation_s": 267.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.3 tok/s {"generation_seconds": 373.189, "generated_tokens": 16176.9}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 245.6, "requests": 21, "tool_calls": 31, "completion_tokens": 5704, "occupied_max": 13986}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 354.4, "requests": 27, "tool_calls": 40, "completion_tokens": 10614, "occupied_max": 25309}] list
tasks_over_64k[] instance ids
completion_tokens16,318 tokens {"finish_reasons": {"tool_calls": 47}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 71 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.7 tok/s, whole run 43.3 tok/s.

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9229192eace16fc1782f479846e0194aea593ba9 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T130251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T131323Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-04 23:13 AEST
Finished2026-10-04 23:22 AEST (8.9 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T125827Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps57.4 tok/s
lab_prefill_tps2,537 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9229192eace16fc1782f479846e0194aea593ba9 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T131323Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2

20261004T132232Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
Started2026-10-04 23:22 AEST
Finished2026-10-04 23:26 AEST (4.2 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds254 s
host_ram_drop_gb64.4 GB
vram_ready_mib20,744 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit97294db30283c264f2c58088bb36bfb1619ba3de dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T132232Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2

20261004T132647Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
Started2026-10-04 23:26 AEST
Finished2026-10-04 23:31 AEST (5.0 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.6 tok/s
lab_prefill_tps2,430 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit97294db30283c264f2c58088bb36bfb1619ba3de dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T132647Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2

20261004T133146Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
Started2026-10-04 23:31 AEST
Finished2026-10-04 23:33 AEST (1.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,014 tok/s {"prompt_tokens": [13689, 13759, 13721], "ttft_s": [6.848, 6.829, 6.812]}
ttft_prefill_32k_tps2,692 tok/s {"prompt_tokens": [54692, 54821, 54730], "ttft_s": [20.316, 20.367, 20.356]}
ttft_prefill_tps2,014 tok/s {"prompt_tokens": [13689, 13759, 13721], "ttft_s": [6.848, 6.829, 6.812]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit97294db30283c264f2c58088bb36bfb1619ba3de dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T133146Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2

20261004T133309Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
Started2026-10-04 23:33 AEST
Finished2026-10-04 23:43 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 57 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.08 < floor 50.0

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max29,027 tokens
context.headroom_min174,880 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls57 calls {"per_min": 5.7, "requests": 42}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps40.1 tok/s
window 1: 42.01 tok/swindow 2: 46.09 tok/swindow 3: 40.09 tok/swindow 4: 43.71 tok/smin 40.09 · max 46.09 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 193, "min_at_generation_s": 181.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.1 tok/s {"generation_seconds": 312.161, "generated_tokens": 13447.2}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 124.1, "requests": 11, "tool_calls": 18, "completion_tokens": 3257, "occupied_max": 8494}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 475.9, "requests": 32, "tool_calls": 39, "completion_tokens": 10303, "occupied_max": 29027}] list
tasks_over_64k[] instance ids
completion_tokens13,560 tokens {"finish_reasons": {"tool_calls": 42}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 57 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 40.1 tok/s, whole run 43.1 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit97294db30283c264f2c58088bb36bfb1619ba3de dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T133309Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2

20261004T134340Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
Started2026-10-04 23:43 AEST
Finished2026-10-04 23:48 AEST (4.6 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps54.6 tok/s
lab_prefill_tps2,580 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit97294db30283c264f2c58088bb36bfb1619ba3de dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T134340Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2

20261004T134832Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14b eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Started2026-10-04 23:48 AEST
Finished2026-10-04 23:52 AEST (3.6 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=pinned
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds215 s
host_ram_drop_gb88.2 GB
vram_ready_mib20,740 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T134832Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2

20261004T135208Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14b eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Started2026-10-04 23:52 AEST
Finished2026-10-04 23:55 AEST (3.7 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=pinned
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps53.7 tok/s
lab_prefill_tps2,440 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T135208Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2

20261004T135547Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T14b eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Started2026-10-04 23:55 AEST
Finished2026-10-04 23:57 AEST (1.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=pinned
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,022 tok/s {"prompt_tokens": [13765, 13739, 13758], "ttft_s": [6.807, 6.819, 6.799]}
ttft_prefill_32k_tps2,700 tok/s {"prompt_tokens": [54781, 54752, 54783], "ttft_s": [20.259, 20.275, 20.301]}
ttft_prefill_tps2,022 tok/s {"prompt_tokens": [13765, 13739, 13758], "ttft_s": [6.807, 6.819, 6.799]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T135547Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2

20261004T135709Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14b eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Started2026-10-04 23:57 AEST
Finished2026-10-05 00:07 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 72 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.915 < floor 50.0

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=pinned
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max39,281 tokens
context.headroom_min165,395 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls72 calls {"per_min": 7.2, "requests": 50}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps40.9 tok/s
window 1: 46.31 tok/swindow 2: 42.86 tok/swindow 3: 45.63 tok/swindow 4: 44.46 tok/swindow 5: 47.12 tok/smin 42.86 · max 47.12 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 289, "min_at_generation_s": 108.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps44.6 tok/s {"generation_seconds": 408.606, "generated_tokens": 18207.3}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 137.5, "requests": 16, "tool_calls": 22, "completion_tokens": 3550, "occupied_max": 10227}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 462.8, "requests": 34, "tool_calls": 50, "completion_tokens": 14781, "occupied_max": 39281}] list
tasks_over_64k[] instance ids
completion_tokens18,331 tokens {"finish_reasons": {"tool_calls": 50}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 72 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 40.9 tok/s, whole run 44.6 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T135709Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2

20261004T140742Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14b eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
Started2026-10-05 00:07 AEST
Finished2026-10-05 00:11 AEST (3.7 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab TTFT stability !

Launch

recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=pinned
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps51.3 tok/s
lab_prefill_tps2,596 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display off
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8off
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T140742Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T141422Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 00:14 AEST
Finished2026-10-05 00:18 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T141422Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds257 s
host_ram_drop_gb64.5 GB
vram_ready_mib20,818 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit742e81596c31f25802bdf11347fc6d9ab1bb8265 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T141422Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261004T141839Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 00:18 AEST
Finished2026-10-05 00:25 AEST (7.0 min)
Exit status0
Notesofficial lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.200k.20261004T142537.json; recipe written only on pass

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T141422Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.8 tok/s
lab_prefill_tps2,419 tok/s "official lab.py try"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit742e81596c31f25802bdf11347fc6d9ab1bb8265 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T141839Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T225552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 08:55 AEST
Finished2026-10-05 09:00 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds260 s
host_ram_drop_gb64.7 GB
vram_ready_mib19,890 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T225552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T230013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 09:00 AEST
Finished2026-10-05 09:04 AEST (4.2 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps48.1 tok/s
lab_prefill_tps2,408 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T230013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T230426Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 09:04 AEST
Finished2026-10-05 09:20 AEST (16.2 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,257 tok/s {"min": 2238.2, "max": 2266.1, "n": 3}
prefill_16k_tps2,207 tok/s {"min": 2201.0, "max": 2213.3, "n": 3}
prefill_32k_tps2,644 tok/s {"min": 2643.9, "max": 2652.5, "n": 3}
prefill_64k_tps2,619 tok/s {"min": 2611.0, "max": 2620.8, "n": 3}
decode_c1_tps43.7 tok/s {"min": 42.8, "max": 44.57, "power_w": null}
decode_c2_tps49.0 tok/s {"min": 48.07, "max": 49.85, "power_w": null}
decode_c3_tps50.8 tok/s {"min": 48.26, "max": 53.38, "power_w": null}
decode_c4_tps49.5 tok/s {"min": 48.16, "max": 50.79, "power_w": null}
decode_c1_32k_tps45.5 tok/s {"min": 45.12, "max": 45.97, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T230426Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T232036Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 09:20 AEST
Finished2026-10-05 09:21 AEST (1.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps1,969 tok/s {"prompt_tokens": [13696, 13698, 13802], "ttft_s": [6.977, 6.955, 6.951]}
ttft_prefill_32k_tps2,650 tok/s {"prompt_tokens": [54741, 54745, 54708], "ttft_s": [20.622, 20.657, 20.663]}
ttft_prefill_tps1,969 tok/s {"prompt_tokens": [13696, 13698, 13802], "ttft_s": [6.977, 6.955, 6.951]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T232036Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T232159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality panel T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 09:21 AEST
Finished2026-10-05 09:22 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.98834 fraction
mean_kl0.00099656 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T232159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T232226Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 09:22 AEST
Finished2026-10-05 09:33 AEST (10.6 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.429 < floor 50.0

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max34,417 tokens
context.headroom_min170,165 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls71 calls {"per_min": 7.1, "requests": 45}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps37.4 tok/s
window 1: 39.3 tok/swindow 2: 40.68 tok/swindow 3: 45.52 tok/swindow 4: 38.23 tok/swindow 5: 41.18 tok/smin 38.23 · max 45.52 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 260, "min_at_generation_s": 263.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps41.1 tok/s {"generation_seconds": 379.256, "generated_tokens": 15573.9}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 255.0, "requests": 22, "tool_calls": 36, "completion_tokens": 6757, "occupied_max": 13743}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 345.0, "requests": 23, "tool_calls": 35, "completion_tokens": 8934, "occupied_max": 34417}] list
tasks_over_64k[] instance ids
completion_tokens15,691 tokens {"finish_reasons": {"tool_calls": 45}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 71 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 37.4 tok/s, whole run 41.1 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T232226Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2

20261004T233302Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
Started2026-10-05 09:33 AEST
Finished2026-10-05 09:37 AEST (4.3 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510baf
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps49.7 tok/s
lab_prefill_tps2,564 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit33d67751778fe52a0111a696912cb037e739d585 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T233302Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261004T233738Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 09:37 AEST
Finished2026-10-05 09:41 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds260 s
host_ram_drop_gb64.3 GB
vram_ready_mib20,818 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T233738Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261004T234159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 09:41 AEST
Finished2026-10-05 09:49 AEST (8.0 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps54.9 tok/s
lab_prefill_tps2,131 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T234159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261004T234959Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 09:49 AEST
Finished2026-10-05 10:05 AEST (15.1 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,270 tok/s {"min": 2269.2, "max": 2290.0, "n": 3}
prefill_16k_tps2,404 tok/s {"min": 2390.5, "max": 2411.1, "n": 3}
prefill_32k_tps2,436 tok/s {"min": 2433.8, "max": 2438.6, "n": 3}
prefill_64k_tps2,424 tok/s {"min": 2423.1, "max": 2429.3, "n": 3}
decode_c1_tps48.2 tok/s {"min": 47.15, "max": 49.33, "power_w": null}
decode_c2_tps53.7 tok/s {"min": 52.77, "max": 54.63, "power_w": null}
decode_c3_tps56.2 tok/s {"min": 55.37, "max": 57.07, "power_w": null}
decode_c4_tps53.1 tok/s {"min": 51.6, "max": 54.52, "power_w": null}
decode_c1_32k_tps48.8 tok/s {"min": 47.71, "max": 49.89, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261004T234959Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T000508Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 10:05 AEST
Finished2026-10-05 10:06 AEST (1.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,294 tok/s {"prompt_tokens": [13720, 13775, 13799], "ttft_s": [6.023, 6.006, 6.007]}
ttft_prefill_32k_tps2,369 tok/s {"prompt_tokens": [54807, 54652, 54690], "ttft_s": [23.113, 23.087, 23.088]}
ttft_prefill_tps2,294 tok/s {"prompt_tokens": [13720, 13775, 13799], "ttft_s": [6.023, 6.006, 6.007]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T000508Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T000636Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality panel T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 10:06 AEST
Finished2026-10-05 10:07 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.98934 fraction
mean_kl0.00091506 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T000636Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T000703Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 10:07 AEST
Finished2026-10-05 10:17 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 64 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.539 < floor 50.0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max25,179 tokens
context.headroom_min178,831 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls64 calls {"per_min": 6.4, "requests": 47}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps39.5 tok/s
window 1: 43.49 tok/swindow 2: 46.17 tok/swindow 3: 44.39 tok/swindow 4: 41.1 tok/smin 41.1 · max 46.17 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 238, "min_at_generation_s": 72.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.9 tok/s {"generation_seconds": 357.957, "generated_tokens": 15726.5}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 202.1, "requests": 22, "tool_calls": 28, "completion_tokens": 5212, "occupied_max": 13779}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 397.9, "requests": 26, "tool_calls": 36, "completion_tokens": 10647, "occupied_max": 25179}] list
tasks_over_64k[] instance ids
completion_tokens15,859 tokens {"finish_reasons": {"tool_calls": 47}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 64 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.5 tok/s, whole run 43.9 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T000703Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T001735Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 10:17 AEST
Finished2026-10-05 10:21 AEST (3.7 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps49.5 tok/s
lab_prefill_tps2,237 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T001735Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261005T002133Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 10:21 AEST
Finished2026-10-05 10:25 AEST (4.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds263 s
host_ram_drop_gb65.3 GB
vram_ready_mib20,822 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitc04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T002133Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261005T002557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 10:25 AEST
Finished2026-10-05 10:32 AEST (6.1 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps49.9 tok/s
lab_prefill_tps2,419 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitc04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T002557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261005T003204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 10:32 AEST
Finished2026-10-05 10:47 AEST (15.2 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,268 tok/s {"min": 2267.6, "max": 2289.9, "n": 3}
prefill_16k_tps2,241 tok/s {"min": 2237.2, "max": 2243.2, "n": 3}
prefill_32k_tps2,669 tok/s {"min": 2668.5, "max": 2679.3, "n": 3}
prefill_64k_tps2,648 tok/s {"min": 2642.8, "max": 2649.4, "n": 3}
decode_c1_tps47.8 tok/s {"min": 46.55, "max": 49.04, "power_w": null}
decode_c2_tps54.5 tok/s {"min": 52.67, "max": 56.29, "power_w": null}
decode_c3_tps55.0 tok/s {"min": 53.94, "max": 56.08, "power_w": null}
decode_c4_tps54.3 tok/s {"min": 52.99, "max": 55.54, "power_w": null}
decode_c1_32k_tps49.7 tok/s {"min": 49.39, "max": 50.06, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitc04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T003204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261005T004714Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 10:47 AEST
Finished2026-10-05 10:48 AEST (1.4 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,021 tok/s {"prompt_tokens": [13841, 13799, 13715], "ttft_s": [6.844, 6.829, 6.841]}
ttft_prefill_32k_tps2,685 tok/s {"prompt_tokens": [54709, 54687, 54693], "ttft_s": [20.355, 20.37, 20.381]}
ttft_prefill_tps2,021 tok/s {"prompt_tokens": [13841, 13799, 13715], "ttft_s": [6.844, 6.829, 6.841]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitc04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T004714Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261005T004836Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality panel T13 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 10:48 AEST
Finished2026-10-05 10:49 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.991 fraction
mean_kl0.00095858 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitc04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T004836Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2

20261005T004903Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✕ crash stability T13 eligibility: none failed stage: measure

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
Started2026-10-05 10:49 AEST
Finished2026-10-05 10:59 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 6 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation)

Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max4,566 tokens
context.headroom_min200,016 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls6 calls {"per_min": 0.6, "requests": 4}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tpsunavailable only 21.565 s of generation; need > 120 s for one window
decode_whole_run_tps43.0 tok/s {"generation_seconds": 21.565, "generated_tokens": 927.9}
tasks_attempted3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomyes {"upstream_connection_failures": 74, "upstream_http_errors": 1, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 303.1, "requests": 4, "tool_calls": 6, "completion_tokens": 942, "occupied_max": 4566}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 261.1}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 35.8}] list
tasks_over_64k[] instance ids
completion_tokens942 tokens {"finish_reasons": {"tool_calls": 4}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 6 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/3, decode min window – tok/s, whole run 43.0 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit99ed822bca74427cce1bd0d05509c9693385510b dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T004903Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T005928Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 10:59 AEST
Finished2026-10-05 11:03 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds257 s
host_ram_drop_gb64.5 GB
vram_ready_mib20,818 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T005928Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T010346Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:03 AEST
Finished2026-10-05 11:10 AEST (7.1 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps53.5 tok/s
lab_prefill_tps2,134 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T010346Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T011051Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:10 AEST
Finished2026-10-05 11:25 AEST (15.1 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,287 tok/s {"min": 2264.6, "max": 2295.6, "n": 3}
prefill_16k_tps2,406 tok/s {"min": 2405.0, "max": 2412.2, "n": 3}
prefill_32k_tps2,440 tok/s {"min": 2439.7, "max": 2445.4, "n": 3}
prefill_64k_tps2,432 tok/s {"min": 2426.3, "max": 2433.4, "n": 3}
decode_c1_tps47.0 tok/s {"min": 46.18, "max": 47.84, "power_w": null}
decode_c2_tps53.5 tok/s {"min": 51.42, "max": 55.58, "power_w": null}
decode_c3_tps56.4 tok/s {"min": 55.16, "max": 57.55, "power_w": null}
decode_c4_tps53.5 tok/s {"min": 51.17, "max": 55.94, "power_w": null}
decode_c1_32k_tps48.3 tok/s {"min": 47.04, "max": 49.6, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T011051Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T012559Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:25 AEST
Finished2026-10-05 11:27 AEST (1.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,283 tok/s {"prompt_tokens": [13674, 13731, 13722], "ttft_s": [6.012, 6.004, 6.01]}
ttft_prefill_32k_tps2,369 tok/s {"prompt_tokens": [54628, 54734, 54799], "ttft_s": [23.071, 23.104, 23.118]}
ttft_prefill_tps2,283 tok/s {"prompt_tokens": [13674, 13731, 13722], "ttft_s": [6.012, 6.004, 6.01]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T012559Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T012727Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality panel T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:27 AEST
Finished2026-10-05 11:27 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.98934 fraction
mean_kl0.00090651 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T012727Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T012753Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:27 AEST
Finished2026-10-05 11:38 AEST (10.6 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 62 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.784 < floor 50.0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max28,209 tokens
context.headroom_min176,303 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls62 calls {"per_min": 6.2, "requests": 49}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps38.8 tok/s
window 1: 41.79 tok/swindow 2: 44.66 tok/swindow 3: 44.2 tok/swindow 4: 39.85 tok/smin 39.85 · max 44.66 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 225, "min_at_generation_s": 247.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.2 tok/s {"generation_seconds": 344.575, "generated_tokens": 14882.0}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 137.4, "requests": 15, "tool_calls": 21, "completion_tokens": 3375, "occupied_max": 10259}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 462.6, "requests": 35, "tool_calls": 42, "completion_tokens": 11629, "occupied_max": 28209}] list
tasks_over_64k[] instance ids
completion_tokens15,004 tokens {"finish_reasons": {"tool_calls": 49}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 62 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 38.8 tok/s, whole run 43.2 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T012753Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T013828Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:38 AEST
Finished2026-10-05 11:45 AEST (7.1 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps56.4 tok/s
lab_prefill_tps2,228 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitd3125c68b10d6f03dba10710bce202f1a355c61f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T013828Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T014552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:45 AEST
Finished2026-10-05 11:50 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds260 s
host_ram_drop_gb65.2 GB
vram_ready_mib20,822 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T014552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T015013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:50 AEST
Finished2026-10-05 11:56 AEST (6.7 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.0 tok/s
lab_prefill_tps2,131 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T015013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T015658Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:56 AEST
Finished2026-10-05 11:58 AEST (1.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,289 tok/s {"prompt_tokens": [13839, 13729, 13726], "ttft_s": [6.033, 6.005, 5.997]}
ttft_prefill_32k_tps2,379 tok/s {"prompt_tokens": [54743, 54741, 54693], "ttft_s": [23.003, 23.011, 23.009]}
ttft_prefill_tps2,289 tok/s {"prompt_tokens": [13839, 13729, 13726], "ttft_s": [6.033, 6.005, 5.997]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T015658Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T015826Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality panel T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:58 AEST
Finished2026-10-05 11:58 AEST (0.4 min)
Exit status0
Notespanel=qwen3.8-flash-next-exl3-ref-panel.json rc=0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement0.99167 fraction
mean_kl0.0009967 nats
in_bandyes "top1>=0.987, KL<=0.0013"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T015826Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T015853Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 11:58 AEST
Finished2026-10-05 12:13 AEST (14.9 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,292 tok/s {"min": 2283.1, "max": 2303.8, "n": 3}
prefill_16k_tps2,400 tok/s {"min": 2395.7, "max": 2402.2, "n": 3}
prefill_32k_tps2,434 tok/s {"min": 2429.8, "max": 2436.1, "n": 3}
prefill_64k_tps2,421 tok/s {"min": 2419.3, "max": 2424.7, "n": 3}
decode_c1_tps48.1 tok/s {"min": 46.11, "max": 50.12, "power_w": null}
decode_c2_tps53.7 tok/s {"min": 51.67, "max": 55.76, "power_w": null}
decode_c3_tps55.7 tok/s {"min": 53.92, "max": 57.53, "power_w": null}
decode_c4_tps53.8 tok/s {"min": 52.2, "max": 55.37, "power_w": null}
decode_c1_32k_tps49.6 tok/s {"min": 48.96, "max": 50.18, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T015853Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T021344Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 12:13 AEST
Finished2026-10-05 12:24 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 61 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.37 < floor 50.0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max30,539 tokens
context.headroom_min174,142 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls61 calls {"per_min": 6.1, "requests": 39}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps40.4 tok/s
window 1: 41.91 tok/swindow 2: 46.32 tok/swindow 3: 43.26 tok/swindow 4: 44.94 tok/swindow 5: 43.94 tok/smin 41.91 · max 46.32 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 252, "min_at_generation_s": 311.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps44.0 tok/s {"generation_seconds": 371.065, "generated_tokens": 16312.6}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 171.3, "requests": 16, "tool_calls": 25, "completion_tokens": 4565, "occupied_max": 10307}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 428.7, "requests": 24, "tool_calls": 36, "completion_tokens": 11840, "occupied_max": 30539}] list
tasks_over_64k[] instance ids
completion_tokens16,405 tokens {"finish_reasons": {"tool_calls": 39}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 61 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 40.4 tok/s, whole run 44.0 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T021344Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T022416Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T14a eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 12:24 AEST
Finished2026-10-05 12:29 AEST (4.9 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps45.0 tok/s
lab_prefill_tps2,240 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89b76f527155eabc290c2443970efffc5809929c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T022416Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T080528Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-load

✓ pass load T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:05 AEST
Finished2026-10-05 18:07 AEST (2.1 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds128 s
host_ram_drop_gb7.43 GB
vram_ready_mib22,038 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitf6465c1b9998f35f8a63466311b53a40b12cb385 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T080528Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-load.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T080737Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab

! fail lab T40 eligibility: none failed stage: gates

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:07 AEST
Finished2026-10-05 18:08 AEST (0.9 min)
Exit status1
Notes–

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
mtp_acceptance_rate0.73 fraction {"mean_accept_len": 3.192, "batches": 4, "source": "SGLang decode log"}
mtp_accepted_tpsunavailable no decode speed in this step
gates_passedunavailable not produced (fail/gates)
lab_decode_c1_tpsunavailable not produced (fail/gates)
lab_prefill_tpsunavailable not produced (fail/gates)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitf6465c1b9998f35f8a63466311b53a40b12cb385 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T080737Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T080832Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-kit_sweep

✓ pass kit sweep T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:08 AEST
Finished2026-10-05 18:21 AEST (12.7 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps1,076 tok/s {"min": 1074.8, "max": 1077.8, "n": 3}
prefill_16k_tps1,038 tok/s {"min": 1037.3, "max": 1041.5, "n": 3}
prefill_32k_tps970 tok/s {"min": 969.7, "max": 970.8, "n": 3}
prefill_64k_tps854 tok/s {"min": 854.0, "max": 854.6, "n": 3}
decode_c1_tps121 tok/s {"min": 119.17, "max": 122.69, "power_w": null}
decode_c2_tps122 tok/s {"min": 120.82, "max": 122.99, "power_w": null}
decode_c3_tps122 tok/s {"min": 120.77, "max": 123.86, "power_w": null}
decode_c4_tps122 tok/s {"min": 121.62, "max": 122.8, "power_w": null}
decode_c1_32k_tps112 tok/s {"min": 106.7, "max": 116.43, "power_w": null}
mtp_acceptance_rate0.5626 fraction {"mean_accept_len": 2.688, "batches": 362, "source": "SGLang decode log"}
mtp_accepted_tps121 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit26d933974ee1425a2fcec22a974c63c651bfa53c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T080832Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-kit_sweep.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T082114Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-ttft

✓ pass TTFT T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:21 AEST
Finished2026-10-05 18:25 AEST (3.8 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps1,035 tok/s {"prompt_tokens": [13746, 13684, 13729], "ttft_s": [13.273, 13.223, 13.268]}
ttft_prefill_32k_tps883 tok/s {"prompt_tokens": [54766, 54807, 54718], "ttft_s": [62.035, 62.071, 61.932]}
ttft_prefill_tps1,035 tok/s {"prompt_tokens": [13746, 13684, 13729], "ttft_s": [13.273, 13.223, 13.268]}
mtp_acceptance_rate0.66 fraction {"mean_accept_len": 2.98, "batches": 1, "source": "SGLang decode log"}
mtp_accepted_tpsunavailable no decode speed in this step

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit26d933974ee1425a2fcec22a974c63c651bfa53c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T082114Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-ttft.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T082500Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-needle

! fail needle T40 eligibility: none failed stage: measure

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:25 AEST
Finished2026-10-05 18:25 AEST (0.1 min)
Exit status0
Noteskit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_max0 tokens
context.headroom_minunavailable no occupancy measured in this run
needle_pass_rate0 fraction [{"target_tokens": 131072, "depth": 0.2, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}, {"target_tokens": 131072, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}, {"target_tokens": 131072, "depth": 0.8, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}, {"target_tokens": 196608, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]
needle_max_passed_tokens0 tokens

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit26d933974ee1425a2fcec22a974c63c651bfa53c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T082500Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-needle.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T082503Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-stability

! fail stability T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:25 AEST
Finished2026-10-05 18:35 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 137 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s)

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_max52,045 tokens
context.headroom_min152,447 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls137 calls {"per_min": 13.7, "requests": 108}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps111 tok/s
window 1: 120.94 tok/swindow 2: 116.4 tok/swindow 3: 113.65 tok/swindow 4: 131.93 tok/swindow 5: 116.65 tok/swindow 6: 113.23 tok/smin 113.23 · max 131.93 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 322, "min_at_generation_s": 342.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps120 tok/s {"generation_seconds": 441.794, "generated_tokens": 52884.0}
tasks_attempted3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved2 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 53.7, "requests": 15, "tool_calls": 20, "completion_tokens": 4610, "occupied_max": 12441}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 304.1, "requests": 44, "tool_calls": 56, "completion_tokens": 25509, "occupied_max": 52045}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 242.5, "requests": 49, "tool_calls": 61, "completion_tokens": 23352, "occupied_max": 35168}] list
tasks_over_64k[] instance ids
completion_tokens53,471 tokens {"finish_reasons": {"tool_calls": 108}}
mtp_acceptance_rate0.6711 fraction {"mean_accept_len": 3.013, "batches": 446, "source": "SGLang decode log"}
mtp_accepted_tps120 tok/s "output tok/s including accepted draft tokens"

Agentic summary: 600 s, 137 tool calls, 0 parse failures, 1 repetition hits, tasks solved 2/3, decode min window 111 tok/s, whole run 120 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit7601cd0b9171268d7ab9b01200decc2680ca4987 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T082503Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-stability.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T083536Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab

! fail lab T40 eligibility: none failed stage: gates

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:35 AEST
Finished2026-10-05 18:36 AEST (0.9 min)
Exit status1
Notes–

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
mtp_acceptance_rate0.71 fraction {"mean_accept_len": 3.123, "batches": 4, "source": "SGLang decode log"}
mtp_accepted_tpsunavailable no decode speed in this step
gates_passedunavailable not produced (fail/gates)
lab_decode_c1_tpsunavailable not produced (fail/gates)
lab_prefill_tpsunavailable not produced (fail/gates)

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit7601cd0b9171268d7ab9b01200decc2680ca4987 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T083536Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2

20261005T083628Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-quality_ev

✓ pass quality eval T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
Started2026-10-05 18:36 AEST
Finished2026-10-05 21:06 AEST (150.0 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !

Launch

recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_max77,769 tokens
context.headroom_min126,604 tokens
tasks_attempted30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved18 tasks {"eval_errors": 0}
solve_rate0.6 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21395, "budget_exhausted": false, "seconds": 302.2, "requests": 38, "tool_calls": 52, "occupied_max": 35932}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19992, "budget_exhausted": false, "seconds": 212.1, "requests": 36, "tool_calls": 40, "occupied_max": 35374}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 54056, "budget_exhausted": false, "seconds": 533.9, "requests": 53, "tool_calls": 69, "occupied_max": 66916}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33030, "budget_exhausted": false, "seconds": 347.7, "requests": 34, "tool_calls": 44, "occupied_max": 56721}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9909, "budget_exhausted": false, "seconds": 102.3, "requests": 24, "tool_calls": 29, "occupied_max": 19157}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27576, "budget_exhausted": false, "seconds": 279.2, "requests": 56, "tool_calls": 69, "occupied_max": 40391}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 4996, "budget_exhausted": false, "seconds": 114.0, "requests": 22, "tool_calls": 32, "occupied_max": 19356}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 16378, "budget_exhausted": false, "seconds": 176.0, "requests": 36, "tool_calls": 36, "occupied_max": 27395}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10825, "budget_exhausted": false, "seconds": 108.7, "requests": 23, "tool_calls": 35, "occupied_max": 24167}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29907, "budget_exhausted": false, "seconds": 331.6, "requests": 49, "tool_calls": 72, "occupied_max": 48615}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 28779, "budget_exhausted": false, "seconds": 252.5, "requests": 27, "tool_calls": 34, "occupied_max": 33418}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 14338, "budget_exhausted": false, "seconds": 168.3, "requests": 40, "tool_calls": 57, "occupied_max": 35704}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 11456, "budget_exhausted": false, "seconds": 129.7, "requests": 30, "tool_calls": 41, "occupied_max": 21732}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 50913, "budget_exhausted": false, "seconds": 657.2, "requests": 79, "tool_calls": 86, "occupied_max": 77769}, {"instance_id": "sympy__sympy-12171", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 506.5, "requests": 33, "tool_calls": 40, "occupied_max": 35466}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 48319, "budget_exhausted": false, "seconds": 543.6, "requests": 80, "tool_calls": 89, "occupied_max": 75240}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7993, "budget_exhausted": false, "seconds": 122.3, "requests": 29, "tool_calls": 32, "occupied_max": 19839}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11280, "budget_exhausted": false, "seconds": 116.3, "requests": 23, "tool_calls": 32, "occupied_max": 22150}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14524, "budget_exhausted": false, "seconds": 232.2, "requests": 47, "tool_calls": 61, "occupied_max": 29338}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 10764, "budget_exhausted": false, "seconds": 121.4, "requests": 34, "tool_calls": 42, "occupied_max": 23921}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 19349, "budget_exhausted": false, "seconds": 210.6, "requests": 45, "tool_calls": 55, "occupied_max": 35457}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 36683, "budget_exhausted": false, "seconds": 377.3, "requests": 57, "tool_calls": 75, "occupied_max": 49046}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25193, "budget_exhausted": false, "seconds": 237.0, "requests": 45, "tool_calls": 52, "occupied_max": 33408}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34857, "budget_exhausted": false, "seconds": 382.7, "requests": 35, "tool_calls": 42, "occupied_max": 41142}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 2815, "budget_exhausted": false, "seconds": 32.9, "requests": 14, "tool_calls": 18, "occupied_max": 7743}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 481.7, "requests": 32, "tool_calls": 38, "occupied_max": 36017}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6628, "budget_exhausted": false, "seconds": 75.5, "requests": 24, "tool_calls": 32, "occupied_max": 14570}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 54124, "budget_exhausted": false, "seconds": 518.1, "requests": 39, "tool_calls": 49, "occupied_max": 57827}, {"instance_id": "django__django-16408", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 575.9, "requests": 49, "tool_calls": 63, "occupied_max": 69131}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24950, "budget_exhausted": false, "seconds": 250.1, "requests": 30, "tool_calls": 37, "occupied_max": 28452}] list
tool_calls1,453 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures5 responses {"wire": {"invalid_tool_call": 4, "unparsed_markup": 1}, "harness_format_errors": {"truncated_no_tool_call": 20, "no_tool_call": 5, "invalid_arguments": 2}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations20 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors5 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits72 hits {"ngram64x3": 72, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}
decode_whole_run_tps125 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records
mtp_acceptance_rate0.7343 fraction {"mean_accept_len": 3.203, "batches": 6353, "source": "SGLang decode log"}
mtp_accepted_tps125 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2bcf6651904c0d286b5235b08ff4e99397b24089 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T083628Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-quality_ev.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T110634Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:06 AEST
Finished2026-10-05 21:10 AEST (4.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds263 s
host_ram_drop_gb64.7 GB
vram_ready_mib20,844 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T110634Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T111058Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:10 AEST
Finished2026-10-05 21:16 AEST (5.3 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps48.8 tok/s
lab_prefill_tps2,131 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T111058Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T111618Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:16 AEST
Finished2026-10-05 21:31 AEST (15.1 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,286 tok/s {"min": 2270.2, "max": 2294.5, "n": 3}
prefill_16k_tps2,408 tok/s {"min": 2397.4, "max": 2413.9, "n": 3}
prefill_32k_tps2,444 tok/s {"min": 2440.5, "max": 2444.1, "n": 3}
prefill_64k_tps2,431 tok/s {"min": 2427.0, "max": 2433.7, "n": 3}
decode_c1_tps47.5 tok/s {"min": 45.23, "max": 49.7, "power_w": null}
decode_c2_tps54.3 tok/s {"min": 52.8, "max": 55.79, "power_w": null}
decode_c3_tps56.2 tok/s {"min": 55.2, "max": 57.26, "power_w": null}
decode_c4_tps54.3 tok/s {"min": 53.1, "max": 55.51, "power_w": null}
decode_c1_32k_tps48.5 tok/s {"min": 47.81, "max": 49.28, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T111618Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T113123Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass TTFT T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:31 AEST
Finished2026-10-05 21:32 AEST (1.5 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,284 tok/s {"prompt_tokens": [13748, 13735, 13713], "ttft_s": [6.019, 6.002, 6.005]}
ttft_prefill_32k_tps2,372 tok/s {"prompt_tokens": [54749, 54796, 54671], "ttft_s": [23.074, 23.105, 23.093]}
ttft_prefill_tps2,284 tok/s {"prompt_tokens": [13748, 13735, 13713], "ttft_s": [6.019, 6.002, 6.005]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T113123Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T113251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail needle T40 eligibility: none failed stage: measure

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:32 AEST
Finished2026-10-05 21:35 AEST (2.3 min)
Exit status0
Noteskit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max104,552 tokens
context.headroom_minunavailable no occupancy measured in this run
needle_pass_rate0.75 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 45.0}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 44.8}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 44.8}, {"target_tokens": 196608, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]
needle_max_passed_tokens104,552 tokens

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T113251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T113507Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:35 AEST
Finished2026-10-05 21:45 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 69 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.78 < floor 50.0

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max28,542 tokens
context.headroom_min175,656 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls69 calls {"per_min": 6.9, "requests": 56}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps39.8 tok/s
window 1: 39.78 tok/swindow 2: 45.05 tok/swindow 3: 46.26 tok/swindow 4: 45.84 tok/smin 39.78 · max 46.26 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 225, "min_at_generation_s": 60.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.6 tok/s {"generation_seconds": 344.08, "generated_tokens": 15015.9}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 198.6, "requests": 20, "tool_calls": 29, "completion_tokens": 5184, "occupied_max": 14275}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 401.4, "requests": 37, "tool_calls": 41, "completion_tokens": 10080, "occupied_max": 29194}] list
tasks_over_64k[] instance ids
completion_tokens15,169 tokens {"finish_reasons": {"tool_calls": 56}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 69 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.8 tok/s, whole run 43.6 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89508b66cd245628d95157a9fb3d38d309518e32 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T113507Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T114539Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:45 AEST
Finished2026-10-05 21:50 AEST (4.5 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps54.6 tok/s
lab_prefill_tps2,234 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89508b66cd245628d95157a9fb3d38d309518e32 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T114539Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T115010Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass quality eval T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-05 21:50 AEST
Finished2026-10-06 03:13 AEST (323.0 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max73,243 tokens
context.headroom_min130,698 tokens
tasks_attempted30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved30 tasks {"eval_errors": 0}
solve_rate1.00 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24487, "budget_exhausted": false, "seconds": 1282.3, "requests": 62, "tool_calls": 75, "occupied_max": 53966}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14321, "budget_exhausted": false, "seconds": 645.2, "requests": 31, "tool_calls": 37, "occupied_max": 28187}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11569, "budget_exhausted": false, "seconds": 520.6, "requests": 26, "tool_calls": 38, "occupied_max": 26802}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20399, "budget_exhausted": false, "seconds": 773.6, "requests": 47, "tool_calls": 56, "occupied_max": 49582}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9370, "budget_exhausted": false, "seconds": 335.5, "requests": 28, "tool_calls": 37, "occupied_max": 21738}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21298, "budget_exhausted": false, "seconds": 893.6, "requests": 50, "tool_calls": 70, "occupied_max": 43815}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9622, "budget_exhausted": false, "seconds": 373.8, "requests": 30, "tool_calls": 42, "occupied_max": 27935}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 38145, "budget_exhausted": false, "seconds": 1384.0, "requests": 67, "tool_calls": 90, "occupied_max": 73243}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9853, "budget_exhausted": false, "seconds": 323.5, "requests": 27, "tool_calls": 41, "occupied_max": 22641}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5844, "budget_exhausted": false, "seconds": 215.4, "requests": 22, "tool_calls": 30, "occupied_max": 13904}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12939, "budget_exhausted": false, "seconds": 472.4, "requests": 43, "tool_calls": 58, "occupied_max": 27930}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10158, "budget_exhausted": false, "seconds": 379.2, "requests": 28, "tool_calls": 39, "occupied_max": 30678}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9242, "budget_exhausted": false, "seconds": 1449.4, "requests": 45, "tool_calls": 54, "occupied_max": 24025}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12532, "budget_exhausted": false, "seconds": 877.7, "requests": 47, "tool_calls": 62, "occupied_max": 34916}, {"instance_id": "sympy__sympy-12171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18740, "budget_exhausted": false, "seconds": 572.2, "requests": 37, "tool_calls": 50, "occupied_max": 38369}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11528, "budget_exhausted": false, "seconds": 398.4, "requests": 39, "tool_calls": 49, "occupied_max": 31537}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15258, "budget_exhausted": false, "seconds": 439.0, "requests": 24, "tool_calls": 28, "occupied_max": 29043}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11438, "budget_exhausted": false, "seconds": 359.5, "requests": 27, "tool_calls": 33, "occupied_max": 21736}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27193, "budget_exhausted": false, "seconds": 911.4, "requests": 63, "tool_calls": 69, "occupied_max": 51234}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11426, "budget_exhausted": false, "seconds": 486.3, "requests": 27, "tool_calls": 41, "occupied_max": 25678}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10873, "budget_exhausted": false, "seconds": 349.6, "requests": 23, "tool_calls": 33, "occupied_max": 27369}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18642, "budget_exhausted": false, "seconds": 806.8, "requests": 42, "tool_calls": 59, "occupied_max": 43584}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22233, "budget_exhausted": false, "seconds": 774.2, "requests": 46, "tool_calls": 60, "occupied_max": 48294}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11156, "budget_exhausted": false, "seconds": 554.6, "requests": 33, "tool_calls": 39, "occupied_max": 27211}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9148, "budget_exhausted": false, "seconds": 321.8, "requests": 24, "tool_calls": 35, "occupied_max": 19416}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26215, "budget_exhausted": false, "seconds": 1133.1, "requests": 56, "tool_calls": 66, "occupied_max": 54318}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6092, "budget_exhausted": false, "seconds": 208.5, "requests": 23, "tool_calls": 26, "occupied_max": 15749}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15049, "budget_exhausted": false, "seconds": 458.2, "requests": 29, "tool_calls": 40, "occupied_max": 25926}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20168, "budget_exhausted": false, "seconds": 735.3, "requests": 44, "tool_calls": 48, "occupied_max": 41643}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10487, "budget_exhausted": false, "seconds": 389.8, "requests": 34, "tool_calls": 40, "occupied_max": 24793}] list
tool_calls1,445 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures1 responses {"wire": {"invalid_tool_call": 1}, "harness_format_errors": {"unknown_tool": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits16 hits {"ngram64x3": 15, "same_call_x4": 1, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}
decode_whole_run_tps42.3 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit89508b66cd245628d95157a9fb3d38d309518e32 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T115010Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T171331Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo

✓ pass load T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:13 AEST
Finished2026-10-06 03:15 AEST (2.1 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T171331Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds128 s
host_ram_drop_gb7.46 GB
vram_ready_mib22,038 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit3c69f81efea146b3536e73d563c29f649bf837c3 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T171331Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T171540Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la

! fail lab T40 eligibility: none failed stage: gates

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:15 AEST
Finished2026-10-06 03:16 AEST (0.9 min)
Exit status1
Notes–

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T171331Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
mtp_acceptance_rate0.7075 fraction {"mean_accept_len": 3.125, "batches": 4, "source": "SGLang decode log"}
mtp_accepted_tpsunavailable no decode speed in this step
gates_passedunavailable not produced (fail/gates)
lab_decode_c1_tpsunavailable not produced (fail/gates)
lab_prefill_tpsunavailable not produced (fail/gates)

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit3c69f81efea146b3536e73d563c29f649bf837c3 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T171540Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T171640Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo

✓ pass load T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:16 AEST
Finished2026-10-06 03:18 AEST (2.1 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T171640Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds128 s
host_ram_drop_gb7.39 GB
vram_ready_mib22,038 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit28a6dff9d14e1e69befacd5bacb1eb12a1e67e16 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T171640Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T171849Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne

! fail needle T40 eligibility: none failed stage: measure

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:18 AEST
Finished2026-10-06 03:25 AEST (7.1 min)
Exit status0
Noteskit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T171640Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_max104,552 tokens
context.headroom_minunavailable no occupancy measured in this run
needle_pass_rate0.75 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 142.1}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 139.7}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 139.9}, {"target_tokens": 180000, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]
needle_max_passed_tokens104,552 tokens

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit28a6dff9d14e1e69befacd5bacb1eb12a1e67e16 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T171849Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T172557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-06 03:25 AEST
Finished2026-10-06 03:30 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T172557Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds260 s
host_ram_drop_gb65.6 GB
vram_ready_mib20,844 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit8458078aea61608e4013c1096b9fda1d2949fa33 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T172557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261005T173018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass needle T40 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-06 03:30 AEST
Finished2026-10-06 03:34 AEST (4.3 min)
Exit status0
Noteskit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T172557Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max191,045 tokens
context.headroom_minunavailable no occupancy measured in this run
needle_pass_rate1.00 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 80.7}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 44.7}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 44.7}, {"target_tokens": 180000, "prompt_tokens": 191045, "depth": 0.5, "code": "5568-7992", "answer": "5568-7992", "pass": true, "seconds": 86.1}]
needle_max_passed_tokens191,045 tokens

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit8458078aea61608e4013c1096b9fda1d2949fa33 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261005T173018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T173533Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo

✓ pass load T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:35 AEST
Finished2026-10-06 03:37 AEST (2.1 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T173533Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds128 s
host_ram_drop_gb7.49 GB
vram_ready_mib22,038 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite435b7aadf7a7467836beef995d22b7c7a6ef9ae dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T173533Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T173741Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la

✓ pass lab T40 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:37 AEST
Finished2026-10-06 03:41 AEST (3.6 min)
Exit status0
Notesserved=turboderp-Qwen3.8-27B-exl3-3.00bpw

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T173533Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps144 tok/s
lab_prefill_tps740 tok/s "context-gate prompt tokens / total request seconds"
mtp_acceptance_rate0.7986 fraction {"mean_accept_len": 3.398, "batches": 77, "source": "SGLang decode log"}
mtp_accepted_tps144 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite435b7aadf7a7467836beef995d22b7c7a6ef9ae dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T173741Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261005T174120Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne

! fail needle T40 eligibility: none failed stage: measure

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-06 03:41 AEST
Finished2026-10-06 03:48 AEST (7.0 min)
Exit status0
Noteskit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261005T173533Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_max104,552 tokens
context.headroom_minunavailable no occupancy measured in this run
needle_pass_rate0.75 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 139.9}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 140.0}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 140.2}, {"target_tokens": 180000, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]
needle_max_passed_tokens104,552 tokens

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite435b7aadf7a7467836beef995d22b7c7a6ef9ae dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261005T174120Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T061419Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load

✓ pass load T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 16:14 AEST
Finished2026-10-08 16:14 AEST (0.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds24.9 s
host_ram_drop_gb39.2 GB
vram_ready_mib23,646 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitee97643caed24b4da8ea676da3e385a7b4c9f6a6 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T061419Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T061444Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab

✓ pass lab T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 16:14 AEST
Finished2026-10-08 16:17 AEST (2.3 min)
Exit status0
Notesserved=qwen3.8-flash-next-q2_0

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps100 tok/s
lab_prefill_tps2,664 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitf47b2f23a90940c82663797750c922ab301f7b72 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T061444Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T061705Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-kit_sweep

! fail kit sweep T41 eligibility: none failed stage: measure

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 16:17 AEST
Finished2026-10-08 16:17 AEST (0.0 min)
Exit status0
Notessweep status: None

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tpsunavailable not reached
prefill_16k_tpsunavailable not reached
prefill_32k_tpsunavailable not reached
prefill_64k_tpsunavailable not reached
decode_c1_tpsunavailable not reached
decode_c2_tpsunavailable not reached
decode_c3_tpsunavailable not reached
decode_c4_tpsunavailable not reached
decode_c1_32k_tpsunavailable not reached
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitf47b2f23a90940c82663797750c922ab301f7b72 clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T061705Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-kit_sweep.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T061706Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-ttft

✓ pass TTFT T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 16:17 AEST
Finished2026-10-08 16:18 AEST (1.3 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,620 tok/s {"prompt_tokens": [13752, 13748, 13713], "ttft_s": [5.421, 5.248, 5.183]}
ttft_prefill_32k_tps2,769 tok/s {"prompt_tokens": [54762, 54790, 54685], "ttft_s": [19.776, 19.785, 19.796]}
ttft_prefill_tps2,620 tok/s {"prompt_tokens": [13752, 13748, 13713], "ttft_s": [5.421, 5.248, 5.183]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit8050e8e736925db26a1dcb695f1a33034f8e7d84 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T061706Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T061821Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-stability

✓ pass stability T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 16:18 AEST
Finished2026-10-08 16:29 AEST (10.6 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 116 tool calls in this run count toward the config's 100-call minimum (§5)

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max38,712 tokens
context.headroom_min92,079 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls116 calls {"per_min": 11.6, "requests": 97}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps123 tok/s
window 1: 128.04 tok/swindow 2: 124.61 tok/swindow 3: 127.52 tok/smin 124.61 · max 128.04 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 151, "min_at_generation_s": 111.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps127 tok/s {"generation_seconds": 270.449, "generated_tokens": 34254.1}
tasks_attempted3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved2 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 52.3, "requests": 18, "tool_calls": 28, "completion_tokens": 3181, "occupied_max": 12800}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 240.9, "requests": 49, "tool_calls": 57, "completion_tokens": 18657, "occupied_max": 38712}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 307.1, "requests": 30, "tool_calls": 31, "completion_tokens": 12531, "occupied_max": 27651}] list
tasks_over_64k[] instance ids
completion_tokens34,369 tokens {"finish_reasons": {"tool_calls": 97}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 116 tool calls, 0 parse failures, 0 repetition hits, tasks solved 2/3, decode min window 123 tok/s, whole run 127 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit8050e8e736925db26a1dcb695f1a33034f8e7d84 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T061821Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-stability.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T062900Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab

✓ pass lab T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 16:29 AEST
Finished2026-10-08 16:32 AEST (3.5 min)
Exit status0
Notesserved=qwen3.8-flash-next-q2_0

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps100 tok/s
lab_prefill_tps2,657 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit8050e8e736925db26a1dcb695f1a33034f8e7d84 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T062900Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2

20261008T063235Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-load

✓ pass load T42 eligibility: none

Configglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
Started2026-10-08 16:32 AEST
Finished2026-10-08 16:34 AEST (1.7 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! kit sweep ! TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds101 s
host_ram_drop_gb51.4 GB
vram_ready_mib22,832 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitadd6123e4cc212c9e92bff5991dcc4be26d9bf9a clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T063235Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-load.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2

20261008T063417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-lab

! fail lab T42 eligibility: none failed stage: gates

Configglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
Started2026-10-08 16:34 AEST
Finished2026-10-08 17:34 AEST (59.7 min)
Exit status0
Notesserved=glm-5.3-flash

Other runs of this config: load lab ! kit sweep ! TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps6.10 tok/s
lab_prefill_tps314 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitadd6123e4cc212c9e92bff5991dcc4be26d9bf9a clean
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T063417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-lab.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2

20261008T073401Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-kit_sweep

! fail kit sweep T42 eligibility: none failed stage: timeout

Configglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
Started2026-10-08 17:34 AEST
Finished2026-10-08 19:34 AEST (120.3 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! kit sweep ! TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tpsunavailable not produced (fail/timeout)
prefill_16k_tpsunavailable not produced (fail/timeout)
prefill_32k_tpsunavailable not produced (fail/timeout)
prefill_64k_tpsunavailable not produced (fail/timeout)
decode_c1_tpsunavailable not produced (fail/timeout)
decode_c2_tpsunavailable not produced (fail/timeout)
decode_c3_tpsunavailable not produced (fail/timeout)
decode_c4_tpsunavailable not produced (fail/timeout)
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitfd5de8462f016c61391e0aacfbaef661ffe68784 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T073401Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-kit_sweep.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2

20261008T093417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-quality_panel

✓ pass quality panel T42 eligibility: none

Configglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
Started2026-10-08 19:34 AEST
Finished2026-10-08 20:25 AEST (51.3 min)
Exit status0
Notespanel=glm-5.3-flash-exl3-ref-panel.json rc=0

Other runs of this config: load lab ! kit sweep ! TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement1.00 fraction
mean_kl0 nats
in_bandyes "top1>=0.99, KL<=0.001"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit4188a3d929072e112d5fc4c196b1f349c8a4f0f8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T093417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-quality_panel.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2

20261008T102534Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-ttft

✓ pass TTFT T42 eligibility: none

Configglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
Started2026-10-08 20:25 AEST
Finished2026-10-08 20:36 AEST (10.7 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab ! kit sweep ! TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps193 tok/s {"prompt_tokens": [10795, 10759, 10750], "ttft_s": [56.03, 50.889, 64.019]}
ttft_prefill_32k_tps278 tok/s {"prompt_tokens": [42966, 42895, 42841], "ttft_s": [166.643, 154.018, 153.111]}
ttft_prefill_tps193 tok/s {"prompt_tokens": [10795, 10759, 10750], "ttft_s": [56.03, 50.889, 64.019]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit4188a3d929072e112d5fc4c196b1f349c8a4f0f8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T102534Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-ttft.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2

20261008T103635Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-load

✓ pass load T42 eligibility: none

Configglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
Started2026-10-08 20:36 AEST
Finished2026-10-08 20:38 AEST (1.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab ! kit sweep TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29c
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds113 s
host_ram_drop_gb88.8 GB
vram_ready_mib22,902 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit6d451232712d567aa4886576429949cd7a6729bf dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T103635Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-load.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2

20261008T103828Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-lab

! fail lab T42 eligibility: none failed stage: gates

Configglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
Started2026-10-08 20:38 AEST
Finished2026-10-08 20:52 AEST (14.3 min)
Exit status0
Notesserved=glm-5.3-flash

Other runs of this config: load lab ! kit sweep TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29c
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_max93,437 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}
lab_decode_c1_tps11.3 tok/s
lab_prefill_tps630 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit6d451232712d567aa4886576429949cd7a6729bf dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T103828Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-lab.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2

20261008T105246Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-kit_sweep

✓ pass kit sweep T42 eligibility: none

Configglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
Started2026-10-08 20:52 AEST
Finished2026-10-08 22:23 AEST (91.1 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load lab ! kit sweep TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29c
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps379 tok/s {"min": 360.7, "max": 381.4, "n": 3}
prefill_16k_tps520 tok/s {"min": 513.0, "max": 520.5, "n": 3}
prefill_32k_tps621 tok/s {"min": 545.1, "max": 633.6, "n": 3}
prefill_64k_tps676 tok/s {"min": 667.4, "max": 677.6, "n": 3}
decode_c1_tps10.9 tok/s {"min": 10.5, "max": 11.39, "power_w": null}
decode_c2_tps11.0 tok/s {"min": 10.57, "max": 11.41, "power_w": null}
decode_c3_tps10.9 tok/s {"min": 10.53, "max": 11.28, "power_w": null}
decode_c4_tps10.5 tok/s {"min": 10.32, "max": 10.74, "power_w": null}
decode_c1_32k_tps10.5 tok/s {"min": 9.6, "max": 11.36, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit6d451232712d567aa4886576429949cd7a6729bf dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T105246Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-kit_sweep.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2

20261008T122352Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-quality_panel

✓ pass quality panel T42 eligibility: none

Configglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
Started2026-10-08 22:23 AEST
Finished2026-10-08 22:26 AEST (2.3 min)
Exit status0
Notespanel=glm-5.3-flash-exl3-ref-panel.json rc=0

Other runs of this config: load lab ! kit sweep TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29c
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
top1_agreement1.00 fraction
mean_kl0 nats
in_bandyes "top1>=0.99, KL<=0.001"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit6d451232712d567aa4886576429949cd7a6729bf dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T122352Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-quality_panel.json

Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2

20261008T122612Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-ttft

✓ pass TTFT T42 eligibility: none

Configglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
Started2026-10-08 22:26 AEST
Finished2026-10-08 22:31 AEST (5.3 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab ! kit sweep TTFT quality panel

Launch

recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Imageghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Digestsha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16
Engineexllamav3
Engine sourcehttps://github.com/0xSero/glm53-flash-offload/tree/7f1ee89
Launch filelaunches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29c
MTP / speculativeoff
argv
envPORT=30000
GLM53_MODE=nvme
GLM53_EC_MAX_SLOTS=1376
GLM53_MAIN_CPUS=1
GLM53_NV_CTL_CPU=16
GLM53_NV_CPU_CPUS=3-15
GLM53_NV_READER_CPUS=19-31
HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3@332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps336 tok/s {"prompt_tokens": [10660, 10631, 10711], "ttft_s": [31.688, 31.731, 31.792]}
ttft_prefill_32k_tps576 tok/s {"prompt_tokens": [42848, 42969, 42779], "ttft_s": [74.448, 80.22, 69.643]}
ttft_prefill_tps336 tok/s {"prompt_tokens": [10660, 10631, 10711], "ttft_s": [31.688, 31.731, 31.792]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit6d451232712d567aa4886576429949cd7a6729bf dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/glm-5.3-flash/20261008T122612Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2

20261008T123143Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T42 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
Started2026-10-08 22:31 AEST
Finished2026-10-08 22:36 AEST (4.3 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab kit sweep stability !

Launch

recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24e
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds260 s
host_ram_drop_gb63.6 GB
vram_ready_mib20,818 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T123143Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2

20261008T123604Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T42 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
Started2026-10-08 22:36 AEST
Finished2026-10-08 22:40 AEST (4.0 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep stability !

Launch

recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24e
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps55.3 tok/s
lab_prefill_tps2,107 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T123604Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2

20261008T124004Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass kit sweep T42 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
Started2026-10-08 22:40 AEST
Finished2026-10-08 22:55 AEST (15.0 min)
Exit status0
Notessweep status: DONE

Other runs of this config: load lab lab kit sweep stability !

Launch

recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24e
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
prefill_8k_tps2,279 tok/s {"min": 2259.9, "max": 2291.3, "n": 3}
prefill_16k_tps2,398 tok/s {"min": 2391.4, "max": 2406.1, "n": 3}
prefill_32k_tps2,435 tok/s {"min": 2434.3, "max": 2437.0, "n": 3}
prefill_64k_tps2,422 tok/s {"min": 2421.1, "max": 2426.8, "n": 3}
decode_c1_tps48.5 tok/s {"min": 47.78, "max": 49.25, "power_w": null}
decode_c2_tps53.5 tok/s {"min": 51.55, "max": 55.47, "power_w": null}
decode_c3_tps55.8 tok/s {"min": 53.66, "max": 57.88, "power_w": null}
decode_c4_tps53.9 tok/s {"min": 53.09, "max": 54.79, "power_w": null}
decode_c1_32k_tps49.1 tok/s {"min": 48.45, "max": 49.82, "power_w": null}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T124004Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2

20261008T125504Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

! fail stability T42 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
Started2026-10-08 22:55 AEST
Finished2026-10-08 23:05 AEST (10.6 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 53 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.954 < floor 50.0

Other runs of this config: load lab lab kit sweep stability !

Launch

recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24e
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max28,256 tokens
context.headroom_min176,327 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls53 calls {"per_min": 5.3, "requests": 34}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps39.0 tok/s
window 1: 42.27 tok/swindow 2: 45.01 tok/swindow 3: 44.23 tok/swindow 4: 39.08 tok/swindow 5: 43.13 tok/swindow 6: 42.37 tok/smin 39.08 · max 45.01 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 332, "min_at_generation_s": 242.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps43.1 tok/s {"generation_seconds": 451.013, "generated_tokens": 19446.5}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 171.2, "requests": 14, "tool_calls": 22, "completion_tokens": 5004, "occupied_max": 11964}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 428.8, "requests": 21, "tool_calls": 31, "completion_tokens": 14526, "occupied_max": 28256}] list
tasks_over_64k[] instance ids
completion_tokens19,530 tokens {"finish_reasons": {"tool_calls": 34}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 53 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.0 tok/s, whole run 43.1 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T125504Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2

20261008T130540Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass lab T42 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
Started2026-10-08 23:05 AEST
Finished2026-10-08 23:08 AEST (2.9 min)
Exit status0
Notesserved=flashnext

Other runs of this config: load lab lab kit sweep stability !

Launch

recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24e
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_max166,667 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps56.1 tok/s
lab_prefill_tps2,222 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T130540Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T132228Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load

✓ pass load T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:22 AEST
Finished2026-10-08 23:23 AEST (0.7 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds43.0 s
host_ram_drop_gb56.9 GB
vram_ready_mib23,660 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T132228Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T132312Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab

✓ pass lab T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:23 AEST
Finished2026-10-08 23:27 AEST (3.9 min)
Exit status0
Notesserved=qwen3.8-flash-next-iq3_s

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps78.1 tok/s
lab_prefill_tps2,174 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T132312Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T132706Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-ttft

✓ pass TTFT T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:27 AEST
Finished2026-10-08 23:28 AEST (1.6 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,036 tok/s {"prompt_tokens": [13659, 13791, 13724], "ttft_s": [6.868, 6.757, 6.742]}
ttft_prefill_32k_tps2,207 tok/s {"prompt_tokens": [54790, 54605, 54677], "ttft_s": [24.701, 24.741, 24.794]}
ttft_prefill_tps2,036 tok/s {"prompt_tokens": [13659, 13791, 13724], "ttft_s": [6.868, 6.757, 6.742]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T132706Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T132842Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-stability

✓ pass stability T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:28 AEST
Finished2026-10-08 23:39 AEST (10.5 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 77 tool calls in this run count toward the config's 100-call minimum (§5)

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max39,624 tokens
context.headroom_min91,304 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls77 calls {"per_min": 7.7, "requests": 54}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps87.3 tok/s
window 1: 87.58 tok/swindow 2: 88.73 tok/smin 87.58 · max 88.73 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 81, "min_at_generation_s": 61.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps85.9 tok/s {"generation_seconds": 200.654, "generated_tokens": 17228.6}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved1 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 87.4, "requests": 13, "tool_calls": 23, "completion_tokens": 3377, "occupied_max": 8278}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 512.8, "requests": 41, "tool_calls": 54, "completion_tokens": 13916, "occupied_max": 39624}] list
tasks_over_64k[] instance ids
completion_tokens17,293 tokens {"finish_reasons": {"tool_calls": 54}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 77 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 87.3 tok/s, whole run 85.9 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T132842Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-stability.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T133915Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab

✓ pass lab T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:39 AEST
Finished2026-10-08 23:42 AEST (3.7 min)
Exit status0
Notesserved=qwen3.8-flash-next-iq3_s

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps75.5 tok/s
lab_prefill_tps2,178 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T133915Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T134305Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load

✓ pass load T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:43 AEST
Finished2026-10-08 23:43 AEST (0.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T134305Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds24.9 s
host_ram_drop_gb40.1 GB
vram_ready_mib23,646 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1baa1b719899523007cf4a95b1ee26102443d216 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T134305Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261008T134331Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval

✓ pass quality eval T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-08 23:43 AEST
Finished2026-10-09 01:40 AEST (117.1 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T134305Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max54,986 tokens
context.headroom_min76,006 tokens
tasks_attempted30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved28 tasks {"eval_errors": 0}
solve_rate0.9333 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14826, "budget_exhausted": false, "seconds": 358.6, "requests": 40, "tool_calls": 45, "occupied_max": 34317}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17039, "budget_exhausted": false, "seconds": 285.5, "requests": 45, "tool_calls": 58, "occupied_max": 36419}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7793, "budget_exhausted": false, "seconds": 155.4, "requests": 20, "tool_calls": 33, "occupied_max": 22226}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15480, "budget_exhausted": false, "seconds": 185.4, "requests": 28, "tool_calls": 40, "occupied_max": 35799}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8323, "budget_exhausted": false, "seconds": 116.7, "requests": 34, "tool_calls": 34, "occupied_max": 20937}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21342, "budget_exhausted": false, "seconds": 377.0, "requests": 59, "tool_calls": 70, "occupied_max": 42618}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7168, "budget_exhausted": false, "seconds": 126.2, "requests": 21, "tool_calls": 30, "occupied_max": 19405}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 10805, "budget_exhausted": false, "seconds": 114.1, "requests": 18, "tool_calls": 26, "occupied_max": 23420}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9637, "budget_exhausted": false, "seconds": 110.4, "requests": 21, "tool_calls": 35, "occupied_max": 24324}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7201, "budget_exhausted": false, "seconds": 90.3, "requests": 23, "tool_calls": 28, "occupied_max": 15577}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7092, "budget_exhausted": false, "seconds": 108.7, "requests": 29, "tool_calls": 33, "occupied_max": 19064}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8009, "budget_exhausted": false, "seconds": 97.4, "requests": 21, "tool_calls": 30, "occupied_max": 22409}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8174, "budget_exhausted": false, "seconds": 214.2, "requests": 29, "tool_calls": 39, "occupied_max": 21183}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15628, "budget_exhausted": false, "seconds": 362.7, "requests": 68, "tool_calls": 79, "occupied_max": 50516}, {"instance_id": "sympy__sympy-12171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11275, "budget_exhausted": false, "seconds": 150.2, "requests": 26, "tool_calls": 35, "occupied_max": 23599}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18094, "budget_exhausted": false, "seconds": 244.5, "requests": 63, "tool_calls": 69, "occupied_max": 46398}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6936, "budget_exhausted": false, "seconds": 78.0, "requests": 17, "tool_calls": 21, "occupied_max": 16200}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11000, "budget_exhausted": false, "seconds": 130.3, "requests": 32, "tool_calls": 34, "occupied_max": 23676}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14879, "budget_exhausted": false, "seconds": 252.9, "requests": 45, "tool_calls": 61, "occupied_max": 34306}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25688, "budget_exhausted": false, "seconds": 517.0, "requests": 55, "tool_calls": 60, "occupied_max": 53147}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12224, "budget_exhausted": false, "seconds": 144.9, "requests": 28, "tool_calls": 38, "occupied_max": 30693}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18211, "budget_exhausted": false, "seconds": 354.9, "requests": 59, "tool_calls": 65, "occupied_max": 46730}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15845, "budget_exhausted": false, "seconds": 273.4, "requests": 41, "tool_calls": 56, "occupied_max": 45996}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29631, "budget_exhausted": false, "seconds": 461.1, "requests": 56, "tool_calls": 63, "occupied_max": 54986}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 1841, "budget_exhausted": false, "seconds": 30.5, "requests": 11, "tool_calls": 14, "occupied_max": 6242}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11918, "budget_exhausted": false, "seconds": 241.8, "requests": 36, "tool_calls": 40, "occupied_max": 34082}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7487, "budget_exhausted": false, "seconds": 93.2, "requests": 30, "tool_calls": 33, "occupied_max": 20098}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 24675, "budget_exhausted": false, "seconds": 448.4, "requests": 48, "tool_calls": 58, "occupied_max": 50684}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22047, "budget_exhausted": false, "seconds": 240.8, "requests": 39, "tool_calls": 46, "occupied_max": 43171}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5504, "budget_exhausted": false, "seconds": 89.3, "requests": 23, "tool_calls": 25, "occupied_max": 15117}] list
tool_calls1,298 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures0 responses {"wire": {}, "harness_format_errors": {"no_tool_call": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors1 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits5 hits {"ngram64x3": 5, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}
decode_whole_run_tps126 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit5a6b28c5783accaacc1c2f0e2472f03355d9caa4 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T134331Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T162803Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load

✓ pass load T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-09 02:28 AEST
Finished2026-10-09 02:28 AEST (0.5 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T162803Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds30.9 s
host_ram_drop_gb56.8 GB
vram_ready_mib23,660 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit5306e4acb0da904fb556961094c6256c16b09f38 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T162803Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261008T162835Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval

✓ pass quality eval T41 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-09 02:28 AEST
Finished2026-10-09 05:40 AEST (192.3 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261008T162803Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max60,497 tokens
context.headroom_min70,102 tokens
tasks_attempted30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved29 tasks {"eval_errors": 0}
solve_rate0.9667 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10130, "budget_exhausted": false, "seconds": 547.2, "requests": 32, "tool_calls": 49, "occupied_max": 26721}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9949, "budget_exhausted": false, "seconds": 258.1, "requests": 23, "tool_calls": 32, "occupied_max": 22571}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13391, "budget_exhausted": false, "seconds": 353.5, "requests": 35, "tool_calls": 49, "occupied_max": 30447}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20221, "budget_exhausted": false, "seconds": 512.2, "requests": 33, "tool_calls": 43, "occupied_max": 54956}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9804, "budget_exhausted": false, "seconds": 183.3, "requests": 24, "tool_calls": 28, "occupied_max": 22437}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21892, "budget_exhausted": false, "seconds": 551.5, "requests": 34, "tool_calls": 51, "occupied_max": 37587}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9860, "budget_exhausted": false, "seconds": 230.6, "requests": 24, "tool_calls": 38, "occupied_max": 29374}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 17676, "budget_exhausted": false, "seconds": 424.8, "requests": 27, "tool_calls": 39, "occupied_max": 32315}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7213, "budget_exhausted": false, "seconds": 131.6, "requests": 20, "tool_calls": 33, "occupied_max": 17051}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10879, "budget_exhausted": false, "seconds": 384.1, "requests": 34, "tool_calls": 47, "occupied_max": 23162}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8532, "budget_exhausted": false, "seconds": 166.0, "requests": 20, "tool_calls": 32, "occupied_max": 17827}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11580, "budget_exhausted": false, "seconds": 268.9, "requests": 30, "tool_calls": 49, "occupied_max": 31286}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10747, "budget_exhausted": false, "seconds": 596.1, "requests": 30, "tool_calls": 50, "occupied_max": 25175}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16507, "budget_exhausted": false, "seconds": 430.6, "requests": 52, "tool_calls": 68, "occupied_max": 55285}, {"instance_id": "sympy__sympy-12171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26317, "budget_exhausted": false, "seconds": 415.9, "requests": 39, "tool_calls": 55, "occupied_max": 49621}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14483, "budget_exhausted": false, "seconds": 568.5, "requests": 56, "tool_calls": 66, "occupied_max": 33074}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7734, "budget_exhausted": false, "seconds": 137.6, "requests": 18, "tool_calls": 24, "occupied_max": 17099}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10270, "budget_exhausted": false, "seconds": 186.6, "requests": 24, "tool_calls": 37, "occupied_max": 20073}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16593, "budget_exhausted": false, "seconds": 356.3, "requests": 46, "tool_calls": 63, "occupied_max": 36246}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17970, "budget_exhausted": false, "seconds": 396.3, "requests": 40, "tool_calls": 56, "occupied_max": 44655}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15975, "budget_exhausted": false, "seconds": 350.9, "requests": 31, "tool_calls": 42, "occupied_max": 35284}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28980, "budget_exhausted": false, "seconds": 627.0, "requests": 63, "tool_calls": 88, "occupied_max": 56435}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27648, "budget_exhausted": false, "seconds": 775.0, "requests": 55, "tool_calls": 83, "occupied_max": 60497}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16028, "budget_exhausted": false, "seconds": 331.5, "requests": 32, "tool_calls": 36, "occupied_max": 28287}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6555, "budget_exhausted": false, "seconds": 133.6, "requests": 17, "tool_calls": 29, "occupied_max": 15328}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19450, "budget_exhausted": false, "seconds": 512.0, "requests": 32, "tool_calls": 50, "occupied_max": 41907}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4607, "budget_exhausted": false, "seconds": 96.0, "requests": 18, "tool_calls": 24, "occupied_max": 13103}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15280, "budget_exhausted": false, "seconds": 374.1, "requests": 25, "tool_calls": 40, "occupied_max": 32560}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12348, "budget_exhausted": false, "seconds": 353.7, "requests": 32, "tool_calls": 41, "occupied_max": 30299}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12660, "budget_exhausted": false, "seconds": 324.1, "requests": 25, "tool_calls": 37, "occupied_max": 24670}] list
tool_calls1,379 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits3 hits {"ngram64x3": 3, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}
decode_whole_run_tps83.5 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit619eb8d354d7d6dbb4bf0039ff6c616d5388cbf2 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261008T162835Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261009T010134Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load

✓ pass load T43 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-09 11:01 AEST
Finished2026-10-09 11:02 AEST (0.5 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261009T010134Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds31.0 s
host_ram_drop_gb56.6 GB
vram_ready_mib23,660 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit901f805673b7fb363b4434146981d5e5dfa5679f dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261009T010134Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261009T010206Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval

✓ pass quality eval T43 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-09 11:02 AEST
Finished2026-10-09 21:36 AEST (633.9 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261009T010134Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max123,185 tokens
context.headroom_min7,250 tokens
tasks_attempted50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved48 tasks {"eval_errors": 1}
solve_rate0.96 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34142, "budget_exhausted": false, "seconds": 795.0, "requests": 75, "tool_calls": 91, "occupied_max": 88574}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40034, "budget_exhausted": false, "seconds": 901.5, "requests": 53, "tool_calls": 66, "occupied_max": 81138}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21957, "budget_exhausted": false, "seconds": 509.4, "requests": 52, "tool_calls": 74, "occupied_max": 59240}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7793, "budget_exhausted": false, "seconds": 124.8, "requests": 13, "tool_calls": 23, "occupied_max": 17264}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17072, "budget_exhausted": false, "seconds": 607.7, "requests": 47, "tool_calls": 69, "occupied_max": 49060}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 46730, "budget_exhausted": false, "seconds": 1276.2, "requests": 87, "tool_calls": 89, "occupied_max": 89048}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18601, "budget_exhausted": false, "seconds": 506.9, "requests": 48, "tool_calls": 63, "occupied_max": 48573}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18441, "budget_exhausted": false, "seconds": 460.1, "requests": 41, "tool_calls": 59, "occupied_max": 41645}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 35736, "budget_exhausted": false, "seconds": 811.1, "requests": 56, "tool_calls": 77, "occupied_max": 70836}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20548, "budget_exhausted": false, "seconds": 352.9, "requests": 24, "tool_calls": 41, "occupied_max": 44792}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17759, "budget_exhausted": false, "seconds": 407.5, "requests": 38, "tool_calls": 47, "occupied_max": 37704}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 41340, "budget_exhausted": false, "seconds": 597.9, "requests": 20, "tool_calls": 29, "occupied_max": 61052}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20349, "budget_exhausted": false, "seconds": 613.8, "requests": 32, "tool_calls": 53, "occupied_max": 41785}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 36273, "budget_exhausted": false, "seconds": 822.7, "requests": 60, "tool_calls": 80, "occupied_max": 77817}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 11846, "budget_exhausted": false, "seconds": 333.6, "requests": 40, "tool_calls": 40, "occupied_max": 34587}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8182, "budget_exhausted": false, "seconds": 221.9, "requests": 14, "tool_calls": 20, "occupied_max": 18033}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18733, "budget_exhausted": false, "seconds": 1487.2, "requests": 58, "tool_calls": 79, "occupied_max": 47859}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12272, "budget_exhausted": false, "seconds": 315.3, "requests": 27, "tool_calls": 38, "occupied_max": 34431}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22608, "budget_exhausted": false, "seconds": 405.7, "requests": 42, "tool_calls": 42, "occupied_max": 44932}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14068, "budget_exhausted": false, "seconds": 264.0, "requests": 28, "tool_calls": 46, "occupied_max": 31724}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40790, "budget_exhausted": false, "seconds": 1774.3, "requests": 95, "tool_calls": 116, "occupied_max": 99465}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 46281, "budget_exhausted": false, "seconds": 1317.5, "requests": 70, "tool_calls": 90, "occupied_max": 85084}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 23745, "budget_exhausted": false, "seconds": 395.1, "requests": 39, "tool_calls": 57, "occupied_max": 46650}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18612, "budget_exhausted": false, "seconds": 495.4, "requests": 40, "tool_calls": 59, "occupied_max": 50057}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18608, "budget_exhausted": false, "seconds": 731.9, "requests": 59, "tool_calls": 78, "occupied_max": 51060}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 45225, "budget_exhausted": false, "seconds": 827.7, "requests": 77, "tool_calls": 108, "occupied_max": 94483}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40896, "budget_exhausted": false, "seconds": 995.6, "requests": 70, "tool_calls": 82, "occupied_max": 77970}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34432, "budget_exhausted": false, "seconds": 1068.6, "requests": 54, "tool_calls": 94, "occupied_max": 74341}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19589, "budget_exhausted": false, "seconds": 337.3, "requests": 39, "tool_calls": 54, "occupied_max": 44753}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6831, "budget_exhausted": false, "seconds": 301.8, "requests": 25, "tool_calls": 43, "occupied_max": 21257}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37119, "budget_exhausted": false, "seconds": 2290.1, "requests": 110, "tool_calls": 146, "occupied_max": 97799}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34047, "budget_exhausted": false, "seconds": 964.8, "requests": 70, "tool_calls": 92, "occupied_max": 78892}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 54771, "budget_exhausted": false, "seconds": 1772.6, "requests": 101, "tool_calls": 103, "occupied_max": 123185}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10761, "budget_exhausted": false, "seconds": 365.1, "requests": 53, "tool_calls": 73, "occupied_max": 34663}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19170, "budget_exhausted": false, "seconds": 480.9, "requests": 46, "tool_calls": 61, "occupied_max": 51197}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18401, "budget_exhausted": false, "seconds": 385.6, "requests": 36, "tool_calls": 56, "occupied_max": 44456}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28022, "budget_exhausted": false, "seconds": 548.1, "requests": 50, "tool_calls": 74, "occupied_max": 66084}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20474, "budget_exhausted": false, "seconds": 1089.8, "requests": 55, "tool_calls": 80, "occupied_max": 52817}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16898, "budget_exhausted": false, "seconds": 542.9, "requests": 27, "tool_calls": 45, "occupied_max": 32301}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13069, "budget_exhausted": false, "seconds": 728.4, "requests": 34, "tool_calls": 50, "occupied_max": 34588}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29334, "budget_exhausted": false, "seconds": 745.8, "requests": 90, "tool_calls": 123, "occupied_max": 72326}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22950, "budget_exhausted": false, "seconds": 789.3, "requests": 40, "tool_calls": 54, "occupied_max": 42461}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14617, "budget_exhausted": false, "seconds": 331.1, "requests": 28, "tool_calls": 49, "occupied_max": 47256}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10457, "budget_exhausted": false, "seconds": 178.2, "requests": 17, "tool_calls": 26, "occupied_max": 23948}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31763, "budget_exhausted": false, "seconds": 783.2, "requests": 65, "tool_calls": 75, "occupied_max": 69969}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 41002, "budget_exhausted": false, "seconds": 1626.9, "requests": 83, "tool_calls": 107, "occupied_max": 107504}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24821, "budget_exhausted": false, "seconds": 1062.5, "requests": 69, "tool_calls": 80, "occupied_max": 64259}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12706, "budget_exhausted": false, "seconds": 265.4, "requests": 23, "tool_calls": 37, "occupied_max": 35352}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 50296, "budget_exhausted": false, "seconds": 1224.1, "requests": 71, "tool_calls": 94, "occupied_max": 91537}, {"instance_id": "roboflow__supervision-2600", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19271, "budget_exhausted": false, "seconds": 414.4, "requests": 24, "tool_calls": 34, "occupied_max": 41350}] list
tool_calls3,366 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures1 responses {"wire": {}, "harness_format_errors": {"truncated_no_tool_call": 1, "missing_command": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations1 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits6 hits {"ngram64x3": 6, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}
decode_whole_run_tps83.0 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit2af53e6e0ae8bbe0782719da7dfce3c7b95379b5 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261009T010206Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261009T113616Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo

✓ pass load T43 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-09 21:36 AEST
Finished2026-10-09 21:38 AEST (2.2 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261009T113616Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds131 s
host_ram_drop_gb7.77 GB
vram_ready_mib22,372 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit678111cd311988184237eefaeef779f3344d0893 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261009T113616Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261009T113828Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-qu

✓ pass quality eval T43 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-09 21:38 AEST
Finished2026-10-10 03:48 AEST (369.6 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261009T113616Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_max86,226 tokens
context.headroom_min112,429 tokens
tasks_attempted50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved36 tasks {"eval_errors": 1}
solve_rate0.72 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 45078, "budget_exhausted": false, "seconds": 657.7, "requests": 73, "tool_calls": 96, "occupied_max": 85404}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 22559, "budget_exhausted": false, "seconds": 426.0, "requests": 46, "tool_calls": 60, "occupied_max": 57351}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26432, "budget_exhausted": false, "seconds": 456.1, "requests": 84, "tool_calls": 94, "occupied_max": 66786}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9552, "budget_exhausted": false, "seconds": 93.7, "requests": 18, "tool_calls": 25, "occupied_max": 20127}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13054, "budget_exhausted": false, "seconds": 277.6, "requests": 38, "tool_calls": 54, "occupied_max": 38748}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 23547, "budget_exhausted": false, "seconds": 561.3, "requests": 46, "tool_calls": 57, "occupied_max": 62365}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13663, "budget_exhausted": false, "seconds": 257.6, "requests": 33, "tool_calls": 45, "occupied_max": 31906}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21920, "budget_exhausted": false, "seconds": 255.6, "requests": 38, "tool_calls": 52, "occupied_max": 37867}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 54531, "budget_exhausted": false, "seconds": 576.0, "requests": 72, "tool_calls": 81, "occupied_max": 76756}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22406, "budget_exhausted": false, "seconds": 244.2, "requests": 31, "tool_calls": 34, "occupied_max": 43755}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14980, "budget_exhausted": false, "seconds": 220.6, "requests": 27, "tool_calls": 30, "occupied_max": 27766}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 55860, "budget_exhausted": false, "seconds": 486.1, "requests": 14, "tool_calls": 10, "occupied_max": 16731}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21555, "budget_exhausted": false, "seconds": 354.8, "requests": 26, "tool_calls": 30, "occupied_max": 28568}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12669, "budget_exhausted": false, "seconds": 170.3, "requests": 34, "tool_calls": 37, "occupied_max": 26755}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 8560, "budget_exhausted": false, "seconds": 193.3, "requests": 24, "tool_calls": 30, "occupied_max": 28471}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22798, "budget_exhausted": false, "seconds": 290.5, "requests": 17, "tool_calls": 23, "occupied_max": 25055}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16220, "budget_exhausted": false, "seconds": 669.1, "requests": 41, "tool_calls": 60, "occupied_max": 41671}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8340, "budget_exhausted": false, "seconds": 477.3, "requests": 35, "tool_calls": 41, "occupied_max": 22159}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 25440, "budget_exhausted": false, "seconds": 288.1, "requests": 37, "tool_calls": 44, "occupied_max": 33085}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24466, "budget_exhausted": false, "seconds": 259.4, "requests": 35, "tool_calls": 50, "occupied_max": 48402}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 42568, "budget_exhausted": false, "seconds": 764.5, "requests": 54, "tool_calls": 61, "occupied_max": 66917}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 627.5, "requests": 56, "tool_calls": 60, "occupied_max": 71198}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 28160, "budget_exhausted": false, "seconds": 245.2, "requests": 25, "tool_calls": 29, "occupied_max": 26916}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11714, "budget_exhausted": false, "seconds": 140.8, "requests": 34, "tool_calls": 40, "occupied_max": 28498}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11519, "budget_exhausted": false, "seconds": 165.3, "requests": 30, "tool_calls": 36, "occupied_max": 27020}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18428, "budget_exhausted": false, "seconds": 244.8, "requests": 49, "tool_calls": 59, "occupied_max": 54832}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 715.7, "requests": 53, "tool_calls": 59, "occupied_max": 82453}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33017, "budget_exhausted": false, "seconds": 418.2, "requests": 46, "tool_calls": 65, "occupied_max": 66120}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 34551, "budget_exhausted": false, "seconds": 345.6, "requests": 67, "tool_calls": 90, "occupied_max": 55591}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5124, "budget_exhausted": false, "seconds": 196.1, "requests": 22, "tool_calls": 33, "occupied_max": 14181}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 2250.1, "requests": 121, "tool_calls": 135, "occupied_max": 82191}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 49347, "budget_exhausted": false, "seconds": 699.2, "requests": 69, "tool_calls": 77, "occupied_max": 76656}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 685.2, "requests": 59, "tool_calls": 98, "occupied_max": 85573}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4433, "budget_exhausted": false, "seconds": 91.7, "requests": 21, "tool_calls": 33, "occupied_max": 16310}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15980, "budget_exhausted": false, "seconds": 222.4, "requests": 43, "tool_calls": 67, "occupied_max": 38851}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18842, "budget_exhausted": false, "seconds": 260.3, "requests": 48, "tool_calls": 65, "occupied_max": 45432}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33864, "budget_exhausted": false, "seconds": 423.4, "requests": 51, "tool_calls": 60, "occupied_max": 56131}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19756, "budget_exhausted": false, "seconds": 613.8, "requests": 55, "tool_calls": 60, "occupied_max": 49105}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15454, "budget_exhausted": false, "seconds": 204.7, "requests": 29, "tool_calls": 38, "occupied_max": 29449}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16226, "budget_exhausted": false, "seconds": 237.5, "requests": 27, "tool_calls": 32, "occupied_max": 36979}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 30651, "budget_exhausted": false, "seconds": 353.4, "requests": 67, "tool_calls": 89, "occupied_max": 61003}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18276, "budget_exhausted": false, "seconds": 403.8, "requests": 48, "tool_calls": 50, "occupied_max": 33827}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8573, "budget_exhausted": false, "seconds": 151.7, "requests": 30, "tool_calls": 43, "occupied_max": 28911}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19383, "budget_exhausted": false, "seconds": 191.7, "requests": 17, "tool_calls": 25, "occupied_max": 31319}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 57933, "budget_exhausted": false, "seconds": 729.1, "requests": 55, "tool_calls": 57, "occupied_max": 68084}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37411, "budget_exhausted": false, "seconds": 648.6, "requests": 77, "tool_calls": 96, "occupied_max": 75095}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 514.9, "requests": 41, "tool_calls": 42, "occupied_max": 35461}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12409, "budget_exhausted": false, "seconds": 146.4, "requests": 28, "tool_calls": 39, "occupied_max": 29397}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 49593, "budget_exhausted": false, "seconds": 773.7, "requests": 59, "tool_calls": 73, "occupied_max": 86226}, {"instance_id": "roboflow__supervision-2600", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": false, "seconds": 596.9, "requests": 19, "tool_calls": 18, "occupied_max": 45787}] list
tool_calls2,682 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures4 responses {"wire": {"invalid_tool_call": 3}, "harness_format_errors": {"truncated_no_tool_call": 39, "no_tool_call": 3, "unknown_tool": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations39 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors3 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits67 hits {"ngram64x3": 67, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}
decode_whole_run_tps119 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records
mtp_acceptance_rate0.7123 fraction {"mean_accept_len": 3.137, "batches": 11337, "source": "SGLang decode log"}
mtp_accepted_tps119 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit678111cd311988184237eefaeef779f3344d0893 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261009T113828Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-qu.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261009T174823Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load

✓ pass load T43 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-10 03:48 AEST
Finished2026-10-10 03:48 AEST (0.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261009T174823Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds24.9 s
host_ram_drop_gb40.1 GB
vram_ready_mib23,646 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite329e6c5aaddf023f1cdd7b407bd8266a2387991 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261009T174823Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261009T174849Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval

✓ pass quality eval T43 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-10 03:48 AEST
Finished2026-10-10 10:22 AEST (393.9 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261009T174823Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max125,121 tokens
context.headroom_min5,323 tokens
tasks_attempted50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved46 tasks {"eval_errors": 1}
solve_rate0.92 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 44538, "budget_exhausted": false, "seconds": 652.8, "requests": 84, "tool_calls": 94, "occupied_max": 97150}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24420, "budget_exhausted": false, "seconds": 488.3, "requests": 51, "tool_calls": 58, "occupied_max": 59275}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18066, "budget_exhausted": false, "seconds": 287.3, "requests": 59, "tool_calls": 68, "occupied_max": 49652}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5949, "budget_exhausted": false, "seconds": 60.0, "requests": 11, "tool_calls": 16, "occupied_max": 13786}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11418, "budget_exhausted": false, "seconds": 293.8, "requests": 47, "tool_calls": 58, "occupied_max": 39569}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31571, "budget_exhausted": false, "seconds": 832.0, "requests": 69, "tool_calls": 75, "occupied_max": 69549}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9136, "budget_exhausted": false, "seconds": 162.1, "requests": 27, "tool_calls": 35, "occupied_max": 30791}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12924, "budget_exhausted": false, "seconds": 268.6, "requests": 31, "tool_calls": 41, "occupied_max": 31169}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21953, "budget_exhausted": false, "seconds": 267.6, "requests": 42, "tool_calls": 48, "occupied_max": 47328}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17697, "budget_exhausted": false, "seconds": 212.0, "requests": 39, "tool_calls": 45, "occupied_max": 40561}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18657, "budget_exhausted": false, "seconds": 262.2, "requests": 23, "tool_calls": 27, "occupied_max": 32044}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 47905, "budget_exhausted": false, "seconds": 553.2, "requests": 34, "tool_calls": 36, "occupied_max": 57301}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9749, "budget_exhausted": false, "seconds": 183.1, "requests": 23, "tool_calls": 24, "occupied_max": 22993}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27497, "budget_exhausted": false, "seconds": 479.9, "requests": 47, "tool_calls": 57, "occupied_max": 54933}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 9342, "budget_exhausted": false, "seconds": 217.1, "requests": 29, "tool_calls": 37, "occupied_max": 31459}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8072, "budget_exhausted": false, "seconds": 106.0, "requests": 12, "tool_calls": 17, "occupied_max": 17177}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10249, "budget_exhausted": false, "seconds": 1395.2, "requests": 47, "tool_calls": 56, "occupied_max": 35318}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4615, "budget_exhausted": false, "seconds": 108.1, "requests": 17, "tool_calls": 21, "occupied_max": 18705}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 12411, "budget_exhausted": false, "seconds": 173.7, "requests": 28, "tool_calls": 34, "occupied_max": 28579}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14133, "budget_exhausted": false, "seconds": 167.8, "requests": 26, "tool_calls": 35, "occupied_max": 33597}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 51570, "budget_exhausted": false, "seconds": 988.2, "requests": 88, "tool_calls": 102, "occupied_max": 113525}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 41744, "budget_exhausted": false, "seconds": 685.9, "requests": 65, "tool_calls": 73, "occupied_max": 80846}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16370, "budget_exhausted": false, "seconds": 196.6, "requests": 43, "tool_calls": 45, "occupied_max": 33487}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15399, "budget_exhausted": false, "seconds": 198.4, "requests": 34, "tool_calls": 50, "occupied_max": 44161}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11607, "budget_exhausted": false, "seconds": 226.1, "requests": 42, "tool_calls": 51, "occupied_max": 28478}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29071, "budget_exhausted": false, "seconds": 400.3, "requests": 63, "tool_calls": 75, "occupied_max": 73574}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 669.9, "requests": 58, "tool_calls": 63, "occupied_max": 110093}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26511, "budget_exhausted": false, "seconds": 436.5, "requests": 65, "tool_calls": 90, "occupied_max": 71966}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31086, "budget_exhausted": false, "seconds": 377.5, "requests": 77, "tool_calls": 87, "occupied_max": 71838}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9729, "budget_exhausted": false, "seconds": 202.4, "requests": 33, "tool_calls": 44, "occupied_max": 25649}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33639, "budget_exhausted": false, "seconds": 2630.1, "requests": 129, "tool_calls": 143, "occupied_max": 81449}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28295, "budget_exhausted": false, "seconds": 764.2, "requests": 99, "tool_calls": 119, "occupied_max": 75551}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1429.1, "requests": 106, "tool_calls": 112, "occupied_max": 125121}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 3606, "budget_exhausted": false, "seconds": 80.0, "requests": 21, "tool_calls": 28, "occupied_max": 14274}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14623, "budget_exhausted": false, "seconds": 211.1, "requests": 35, "tool_calls": 45, "occupied_max": 40292}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9638, "budget_exhausted": false, "seconds": 133.2, "requests": 26, "tool_calls": 38, "occupied_max": 31805}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26209, "budget_exhausted": false, "seconds": 318.9, "requests": 38, "tool_calls": 49, "occupied_max": 54186}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18365, "budget_exhausted": false, "seconds": 572.4, "requests": 39, "tool_calls": 59, "occupied_max": 56748}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16460, "budget_exhausted": false, "seconds": 427.6, "requests": 35, "tool_calls": 40, "occupied_max": 30798}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12223, "budget_exhausted": false, "seconds": 260.6, "requests": 30, "tool_calls": 40, "occupied_max": 34837}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16756, "budget_exhausted": false, "seconds": 363.2, "requests": 52, "tool_calls": 69, "occupied_max": 46110}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12158, "budget_exhausted": false, "seconds": 385.1, "requests": 30, "tool_calls": 33, "occupied_max": 22195}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8741, "budget_exhausted": false, "seconds": 210.8, "requests": 30, "tool_calls": 39, "occupied_max": 29740}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8337, "budget_exhausted": false, "seconds": 96.0, "requests": 22, "tool_calls": 22, "occupied_max": 18404}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37422, "budget_exhausted": false, "seconds": 495.7, "requests": 56, "tool_calls": 58, "occupied_max": 70022}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31749, "budget_exhausted": false, "seconds": 1069.9, "requests": 76, "tool_calls": 84, "occupied_max": 80814}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14637, "budget_exhausted": false, "seconds": 307.2, "requests": 48, "tool_calls": 54, "occupied_max": 44288}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6244, "budget_exhausted": false, "seconds": 93.8, "requests": 21, "tool_calls": 26, "occupied_max": 20934}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34275, "budget_exhausted": false, "seconds": 734.5, "requests": 71, "tool_calls": 96, "occupied_max": 68587}, {"instance_id": "roboflow__supervision-2600", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22526, "budget_exhausted": false, "seconds": 297.8, "requests": 28, "tool_calls": 32, "occupied_max": 42258}] list
tool_calls2,748 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures0 responses {"wire": {}, "harness_format_errors": {"truncated_no_tool_call": 3, "no_tool_call": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations3 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors1 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits7 hits {"ngram64x3": 6, "same_call_x4": 1, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}
decode_whole_run_tps123 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite329e6c5aaddf023f1cdd7b407bd8266a2387991 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261009T174849Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261010T002303Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T43 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-10 10:23 AEST
Finished2026-10-10 10:27 AEST (4.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T002303Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds267 s
host_ram_drop_gb65.1 GB
vram_ready_mib20,844 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit29c8bc5c793d774d8c9cb4d73dba668b9d156223 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T002303Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261010T104600Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load

✓ pass load T44 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-10 20:46 AEST
Finished2026-10-10 20:46 AEST (0.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T104600Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds25.1 s
host_ram_drop_gb40.0 GB
vram_ready_mib23,646 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitca980ac49e0fa5fad31e9e3a3b3c32c4ec092e7e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T104600Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2

20261010T104626Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-resource_profile

✓ pass resources T44 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
Started2026-10-10 20:46 AEST
Finished2026-10-10 20:48 AEST (1.8 min)
Exit status0
Notesresource profile; GPU index 2; probe 1065 samples

Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T104600Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
idle_vram_used_gib_max23.5 GiB
idle_cpu_cores_busy_mean0.1 cores
idle_cpu_cores_busy_p950.27 cores
idle_gpu_util_mean_pct0 %
idle_pcie_rx_gbps_mean0 GB/s
idle_power_w_mean100 W
idle_ram_anon_gib33.2 GiB "container cgroup memory.stat at phase end"
idle_ram_file_gib39.1 GiB "container cgroup memory.stat at phase end"
prefill_vram_used_gib_max23.6 GiB
prefill_cpu_cores_busy_mean1.33 cores
prefill_cpu_cores_busy_p952.88 cores
prefill_gpu_util_mean_pct93.4 %
prefill_pcie_rx_gbps_mean8.42 GB/s
prefill_power_w_mean348 W
prefill_ram_anon_gib33.6 GiB "container cgroup memory.stat at phase end"
prefill_ram_file_gib39.1 GiB "container cgroup memory.stat at phase end"
decode_vram_used_gib_max23.6 GiB
decode_cpu_cores_busy_mean15.2 cores
decode_cpu_cores_busy_p9515.6 cores
decode_gpu_util_mean_pct99.6 %
decode_pcie_rx_gbps_mean0.78 GB/s
decode_power_w_mean381 W
decode_ram_anon_gib33.6 GiB "container cgroup memory.stat at phase end"
decode_ram_file_gib39.1 GiB "container cgroup memory.stat at phase end"
prefill_tps_median2,584 tok/s {"prompt_tokens": [26958, 27045, 27018]}
decode_tps_median123 tok/s {"completion_tokens": [2007, 2047, 2048]}

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitca980ac49e0fa5fad31e9e3a3b3c32c4ec092e7e dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T104626Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-resource_profile.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261010T104819Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load

✓ pass load T44 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-10 20:48 AEST
Finished2026-10-10 20:48 AEST (0.5 min)
Exit statusNone
Notes–

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T104819Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds31.1 s
host_ram_drop_gb56.9 GB
vram_ready_mib23,660 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitdbbb8ed9b3962d9817eca755993dbac443c161e8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T104819Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2

20261010T104851Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-resource_profile

✓ pass resources T44 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
Started2026-10-10 20:48 AEST
Finished2026-10-10 20:50 AEST (2.1 min)
Exit status0
Notesresource profile; GPU index 2; probe 1264 samples

Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T104819Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
idle_vram_used_gib_max23.6 GiB
idle_cpu_cores_busy_mean0.19 cores
idle_cpu_cores_busy_p950.29 cores
idle_gpu_util_mean_pct0 %
idle_pcie_rx_gbps_mean0 GB/s
idle_power_w_mean109 W
idle_ram_anon_gib48.4 GiB "container cgroup memory.stat at phase end"
idle_ram_file_gib55.2 GiB "container cgroup memory.stat at phase end"
prefill_vram_used_gib_max23.6 GiB
prefill_cpu_cores_busy_mean1.29 cores
prefill_cpu_cores_busy_p952.60 cores
prefill_gpu_util_mean_pct95.6 %
prefill_pcie_rx_gbps_mean11.6 GB/s
prefill_power_w_mean322 W
prefill_ram_anon_gib48.9 GiB "container cgroup memory.stat at phase end"
prefill_ram_file_gib55.2 GiB "container cgroup memory.stat at phase end"
decode_vram_used_gib_max23.7 GiB
decode_cpu_cores_busy_mean15.2 cores
decode_cpu_cores_busy_p9515.6 cores
decode_gpu_util_mean_pct99.6 %
decode_pcie_rx_gbps_mean3.47 GB/s
decode_power_w_mean359 W
decode_ram_anon_gib48.8 GiB "container cgroup memory.stat at phase end"
decode_ram_file_gib55.2 GiB "container cgroup memory.stat at phase end"
prefill_tps_median1,964 tok/s {"prompt_tokens": [26991, 27006, 26963]}
decode_tps_median93.8 tok/s {"completion_tokens": [1975, 1806, 1812]}

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit1ccc5e213b6654f6329de7c5db4e5183fe6b92c8 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T104851Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-resource_profile.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261010T105104Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo

✓ pass load T44 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-10 20:51 AEST
Finished2026-10-10 20:53 AEST (2.2 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T105104Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds131 s
host_ram_drop_gb7.63 GB
vram_ready_mib22,374 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9035401e49a076325efdade467af0c5ddb3d62ca dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261010T105104Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2

20261010T105317Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-re

✓ pass resources T44 eligibility: none

Configsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
Started2026-10-10 20:53 AEST
Finished2026-10-10 20:55 AEST (2.6 min)
Exit status0
Notesresource profile; GPU index 2; probe 1569 samples

Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T105104Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Digestsha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91b
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
idle_vram_used_gib_max22.3 GiB
idle_cpu_cores_busy_mean1.10 cores
idle_cpu_cores_busy_p951.38 cores
idle_gpu_util_mean_pct0 %
idle_pcie_rx_gbps_mean0 GB/s
idle_power_w_mean113 W
idle_ram_anon_gib5.44 GiB "container cgroup memory.stat at phase end"
idle_ram_file_gib17.1 GiB "container cgroup memory.stat at phase end"
prefill_vram_used_gib_max22.6 GiB
prefill_cpu_cores_busy_mean1.10 cores
prefill_cpu_cores_busy_p951.27 cores
prefill_gpu_util_mean_pct98.8 %
prefill_pcie_rx_gbps_mean0.18 GB/s
prefill_power_w_mean384 W
prefill_ram_anon_gib5.37 GiB "container cgroup memory.stat at phase end"
prefill_ram_file_gib17.1 GiB "container cgroup memory.stat at phase end"
decode_vram_used_gib_max22.6 GiB
decode_cpu_cores_busy_mean1.11 cores
decode_cpu_cores_busy_p951.28 cores
decode_gpu_util_mean_pct95.3 %
decode_pcie_rx_gbps_mean2.06 GB/s
decode_power_w_mean388 W
decode_ram_anon_gib5.37 GiB "container cgroup memory.stat at phase end"
decode_ram_file_gib17.1 GiB "container cgroup memory.stat at phase end"
prefill_tps_median984 tok/s {"prompt_tokens": [26936, 27003, 26963]}
decode_tps_median108 tok/s {"completion_tokens": [2008, 1773, 1793]}
mtp_acceptance_rate0.4751 fraction {"mean_accept_len": 2.425, "batches": 57, "source": "SGLang decode log"}
mtp_accepted_tpsunavailable no decode speed in this step

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit9035401e49a076325efdade467af0c5ddb3d62ca dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261010T105317Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-re.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261010T105600Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass load T44 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-10 20:56 AEST
Finished2026-10-10 21:00 AEST (4.4 min)
Exit statusNone
Notes–

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T105600Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds264 s
host_ram_drop_gb64.2 GB
vram_ready_mib20,818 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitb37af885cdaa897b50aec9052c890862cfc967c5 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T105600Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2

20261010T110024Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe

✓ pass resources T44 eligibility: none

Configsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Started2026-10-10 21:00 AEST
Finished2026-10-10 21:03 AEST (3.2 min)
Exit status0
Notesresource profile; GPU index 2; probe 1900 samples

Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261010T105600Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
Imageghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Digestsha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163
Enginesglang
Engine sourcehttps://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a
Launch filelaunches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587
MTP / speculativeoff
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder
envHF_HUB_OFFLINE=1
CUDA_DEVICE_ORDER=PCI_BUS_ID
SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5
SGLANG_EXL3_MOE_OFFLOAD=gpu_cache
EXL3_MOE_CPU_THREADS=24
SGLANG_EXL3_EXPERT_CACHE_GB=auto
SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26
SGLANG_EXL3_KV_BITS=5
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4
SGLANG_EXL3_OFFLOAD_FUSED=1
SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1
SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1
SGLANG_EXL3_NGRAM_TIER=nvme
SGLANG_EXL3_VISION=1
SGLANG_EXL3_MM_FAST_CPU=1
SGLANG_EXL3_VIT_SDPA=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3@69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json

Metrics

context.configured204,800 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
idle_vram_used_gib_max21.1 GiB
idle_cpu_cores_busy_mean1.20 cores
idle_cpu_cores_busy_p952.10 cores
idle_gpu_util_mean_pct1.30 %
idle_pcie_rx_gbps_mean0.05 GB/s
idle_power_w_mean152 W
idle_ram_anon_gib57.2 GiB "container cgroup memory.stat at phase end"
idle_ram_file_gib54.8 GiB "container cgroup memory.stat at phase end"
prefill_vram_used_gib_max22.9 GiB
prefill_cpu_cores_busy_mean1.28 cores
prefill_cpu_cores_busy_p951.81 cores
prefill_gpu_util_mean_pct74.5 %
prefill_pcie_rx_gbps_mean9.19 GB/s
prefill_power_w_mean279 W
prefill_ram_anon_gib57.5 GiB "container cgroup memory.stat at phase end"
prefill_ram_file_gib55.5 GiB "container cgroup memory.stat at phase end"
decode_vram_used_gib_max22.9 GiB
decode_cpu_cores_busy_mean1.26 cores
decode_cpu_cores_busy_p951.61 cores
decode_gpu_util_mean_pct99.6 %
decode_pcie_rx_gbps_mean6.38 GB/s
decode_power_w_mean309 W
decode_ram_anon_gib57.4 GiB "container cgroup memory.stat at phase end"
decode_ram_file_gib55.8 GiB "container cgroup memory.stat at phase end"
prefill_tps_median2,131 tok/s {"prompt_tokens": [26967, 26935, 26998]}
decode_tps_median53.0 tok/s {"completion_tokens": [2039, 1749, 2048]}

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitb37af885cdaa897b50aec9052c890862cfc967c5 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261010T110024Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2

20261011T004748Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-load

✓ pass load T45 eligibility: none

Configstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2
Started2026-10-11 10:47 AEST
Finished2026-10-11 10:48 AEST (0.5 min)
Exit statusNone
Notes–

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261011T004748Z-lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090- --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=coder -e MODEL=IQ1_M -e GGUF_DIR=/gguf/IQ1_M -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb.json sha256 21c445ce51f66e2e7e99721e02a761b0c7b3ef4c7d39c78392fc71a6361126b7
MTP / speculativeoff
argv
envFAMILY=coder
MODEL=IQ1_M
GGUF_DIR=/gguf/IQ1_M
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF@5348543e0147355ac9cbcb031184a3546350988e · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds28.1 s
host_ram_drop_gb31.0 GB
vram_ready_mib23,662 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit13a7ca5bf6699db3c4f3d83874a810411069e2e4 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T004748Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2

20261011T004817Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-lab

✓ pass lab T45 eligibility: none

Configstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2
Started2026-10-11 10:48 AEST
Finished2026-10-11 10:50 AEST (2.4 min)
Exit status0
Notesserved=qwen3.8-flash-next-coder-iq1_m

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261011T004748Z-lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090- --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=coder -e MODEL=IQ1_M -e GGUF_DIR=/gguf/IQ1_M -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb.json sha256 21c445ce51f66e2e7e99721e02a761b0c7b3ef4c7d39c78392fc71a6361126b7
MTP / speculativeoff
argv
envFAMILY=coder
MODEL=IQ1_M
GGUF_DIR=/gguf/IQ1_M
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF@5348543e0147355ac9cbcb031184a3546350988e · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps93.2 tok/s
lab_prefill_tps2,657 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit13a7ca5bf6699db3c4f3d83874a810411069e2e4 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T004817Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2

20261011T005041Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-ttft

✓ pass TTFT T45 eligibility: none

Configstrata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2
Started2026-10-11 10:50 AEST
Finished2026-10-11 10:51 AEST (1.3 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab TTFT

Launch

recorded by tools/run.py (artifacts/runs/20261011T004748Z-lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090- --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=coder -e MODEL=IQ1_M -e GGUF_DIR=/gguf/IQ1_M -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb.json sha256 21c445ce51f66e2e7e99721e02a761b0c7b3ef4c7d39c78392fc71a6361126b7
MTP / speculativeoff
argv
envFAMILY=coder
MODEL=IQ1_M
GGUF_DIR=/gguf/IQ1_M
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF@5348543e0147355ac9cbcb031184a3546350988e · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,639 tok/s {"prompt_tokens": [13760, 13727, 13703], "ttft_s": [5.413, 5.202, 5.171]}
ttft_prefill_32k_tps2,761 tok/s {"prompt_tokens": [54631, 54664, 54748], "ttft_s": [19.784, 19.8, 19.857]}
ttft_prefill_tps2,639 tok/s {"prompt_tokens": [13760, 13727, 13703], "ttft_s": [5.413, 5.202, 5.171]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit13a7ca5bf6699db3c4f3d83874a810411069e2e4 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T005041Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-ttft.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T014733Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass load T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 11:47 AEST
Finished2026-10-11 11:48 AEST (0.6 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds34.1 s
host_ram_drop_gb59.9 GB
vram_ready_mib23,656 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbe97d5d0e733b936a850c94c188eefc4666db617 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T014733Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T014808Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass lab T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 11:48 AEST
Finished2026-10-11 11:55 AEST (7.0 min)
Exit status0
Notesserved=orcarouter-qwen3.8-flash-next-uncensored-iq3_xxs

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps76.4 tok/s
lab_prefill_tps2,122 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbe97d5d0e733b936a850c94c188eefc4666db617 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T014808Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T015508Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass TTFT T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 11:55 AEST
Finished2026-10-11 11:56 AEST (1.6 min)
Exit status0
Notesheadline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
ttft_prefill_8k_tps2,007 tok/s {"prompt_tokens": [13831, 13746, 13733], "ttft_s": [7.019, 6.827, 6.843]}
ttft_prefill_32k_tps2,162 tok/s {"prompt_tokens": [54799, 54714, 54691], "ttft_s": [25.269, 25.307, 25.318]}
ttft_prefill_tps2,007 tok/s {"prompt_tokens": [13831, 13746, 13733], "ttft_s": [7.019, 6.827, 6.843]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbe97d5d0e733b936a850c94c188eefc4666db617 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T015508Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T015645Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

! fail stability T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 11:56 AEST
Finished2026-10-11 12:06 AEST (10.1 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 50 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 2 repetition hit(s)

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max56,311 tokens
context.headroom_min73,648 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls50 calls {"per_min": 5.0, "requests": 35}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits2 hits {"ngram64x3": 2, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps86.0 tok/s
window 1: 90.26 tok/swindow 2: 94.88 tok/swindow 3: 90.89 tok/swindow 4: 96.59 tok/swindow 5: 93.48 tok/swindow 6: 90.89 tok/swindow 7: 87.54 tok/smin 87.54 · max 96.59 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 383, "min_at_generation_s": 412.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps91.3 tok/s {"generation_seconds": 502.698, "generated_tokens": 45910.6}
tasks_attempted2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved0 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": false, "resolved": false, "seconds": 41.8, "requests": 11, "tool_calls": 17, "completion_tokens": 1647, "occupied_max": 5130}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 558.2, "requests": 25, "tool_calls": 33, "completion_tokens": 44305, "occupied_max": 56311}] list
tasks_over_64k[] instance ids
completion_tokens45,952 tokens {"finish_reasons": {"tool_calls": 35}}
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Agentic summary: 600 s, 50 tool calls, 0 parse failures, 2 repetition hits, tasks solved 0/2, decode min window 86.0 tok/s, whole run 91.3 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbe97d5d0e733b936a850c94c188eefc4666db617 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T015645Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T020654Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass lab T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 12:06 AEST
Finished2026-10-11 12:11 AEST (4.6 min)
Exit status0
Notesserved=orcarouter-qwen3.8-flash-next-uncensored-iq3_xxs

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max106,294 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps73.6 tok/s
lab_prefill_tps2,105 tok/s "context-gate prompt tokens / total request seconds"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbe97d5d0e733b936a850c94c188eefc4666db617 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T020654Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T021133Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass resources T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 12:11 AEST
Finished2026-10-11 12:13 AEST (2.0 min)
Exit status0
Notesresource profile; GPU index 2; probe 1218 samples

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
idle_vram_used_gib_max23.7 GiB
idle_cpu_cores_busy_mean0.09 cores
idle_cpu_cores_busy_p950.28 cores
idle_gpu_util_mean_pct0 %
idle_pcie_rx_gbps_mean0 GB/s
idle_power_w_mean120 W
idle_ram_anon_gib51.8 GiB "container cgroup memory.stat at phase end"
idle_ram_file_gib57.2 GiB "container cgroup memory.stat at phase end"
prefill_vram_used_gib_max23.7 GiB
prefill_cpu_cores_busy_mean1.34 cores
prefill_cpu_cores_busy_p952.58 cores
prefill_gpu_util_mean_pct95.3 %
prefill_pcie_rx_gbps_mean12.5 GB/s
prefill_power_w_mean320 W
prefill_ram_anon_gib51.9 GiB "container cgroup memory.stat at phase end"
prefill_ram_file_gib57.2 GiB "container cgroup memory.stat at phase end"
decode_vram_used_gib_max23.7 GiB
decode_cpu_cores_busy_mean15.2 cores
decode_cpu_cores_busy_p9515.6 cores
decode_gpu_util_mean_pct99.9 %
decode_pcie_rx_gbps_mean3.81 GB/s
decode_power_w_mean352 W
decode_ram_anon_gib51.8 GiB "container cgroup memory.stat at phase end"
decode_ram_file_gib57.2 GiB "container cgroup memory.stat at phase end"
prefill_tps_median1,911 tok/s {"prompt_tokens": [26972, 26947, 26949]}
decode_tps_median92.8 tok/s {"completion_tokens": [1788, 1603, 1584]}

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commitbe97d5d0e733b936a850c94c188eefc4666db617 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T021133Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T021336Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass quality eval T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-11 12:13 AEST
Finished2026-10-12 00:41 AEST (748.0 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max127,837 tokens
context.headroom_min257 tokens
tasks_attempted50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved14 tasks {"eval_errors": 0}
solve_rate0.28 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 919.2, "requests": 57, "tool_calls": 79, "occupied_max": 127837}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": false, "seconds": 720.2, "requests": 24, "tool_calls": 28, "occupied_max": 42929}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 47816, "budget_exhausted": false, "seconds": 900.3, "requests": 65, "tool_calls": 91, "occupied_max": 92029}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24615, "budget_exhausted": false, "seconds": 346.3, "requests": 22, "tool_calls": 34, "occupied_max": 40074}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18073, "budget_exhausted": false, "seconds": 948.4, "requests": 60, "tool_calls": 77, "occupied_max": 51047}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1069.6, "requests": 53, "tool_calls": 84, "occupied_max": 112191}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 816.9, "requests": 37, "tool_calls": 58, "occupied_max": 85619}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 880.4, "requests": 39, "tool_calls": 60, "occupied_max": 79018}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 53359, "budget_exhausted": false, "seconds": 765.0, "requests": 43, "tool_calls": 51, "occupied_max": 73796}, {"instance_id": "mesa__mesa-3892", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 775.9, "requests": 27, "tool_calls": 30, "occupied_max": 59330}, {"instance_id": "pytorch__rl-4210", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 757.1, "requests": 27, "tool_calls": 35, "occupied_max": 51443}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 23221, "budget_exhausted": false, "seconds": 281.7, "requests": 9, "tool_calls": 9, "occupied_max": 17707}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25561, "budget_exhausted": false, "seconds": 839.3, "requests": 39, "tool_calls": 54, "occupied_max": 48657}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 960.6, "requests": 39, "tool_calls": 55, "occupied_max": 68606}, {"instance_id": "domokane__FinancePy-317", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 929.3, "requests": 46, "tool_calls": 59, "occupied_max": 89011}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 52659, "budget_exhausted": false, "seconds": 691.1, "requests": 23, "tool_calls": 30, "occupied_max": 47569}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22123, "budget_exhausted": false, "seconds": 2276.9, "requests": 67, "tool_calls": 82, "occupied_max": 57552}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13455, "budget_exhausted": false, "seconds": 387.3, "requests": 28, "tool_calls": 42, "occupied_max": 33183}, {"instance_id": "docling-project__docling-4318", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 749.8, "requests": 34, "tool_calls": 33, "occupied_max": 55204}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 56528, "budget_exhausted": false, "seconds": 837.3, "requests": 47, "tool_calls": 70, "occupied_max": 77867}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Error:BadRequestError: litellm.BadRequestError: OpenAIException - prompt (126342 tokens) + max tokens (8192) exceeds the context (131072); requests are never truncated. Send a smaller max_tokens (at most 4722 here), or add \"fit_max_tokens\": true to the model's strata-<model>.json to shorten it to th", "submitted": false, "resolved": false, "completion_tokens": 51783, "budget_exhausted": false, "seconds": 1127.3, "requests": 60, "tool_calls": 106, "occupied_max": 121042}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 753.4, "requests": 22, "tool_calls": 38, "occupied_max": 73371}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 962.7, "requests": 46, "tool_calls": 55, "occupied_max": 78233}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 32200, "budget_exhausted": false, "seconds": 629.9, "requests": 44, "tool_calls": 71, "occupied_max": 73471}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1650.9, "requests": 74, "tool_calls": 113, "occupied_max": 113230}, {"instance_id": "marin-community__marin-9450", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 858.3, "requests": 37, "tool_calls": 71, "occupied_max": 101082}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 685.2, "requests": 22, "tool_calls": 28, "occupied_max": 54267}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 51535, "budget_exhausted": false, "seconds": 946.1, "requests": 72, "tool_calls": 102, "occupied_max": 100772}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 947.1, "requests": 94, "tool_calls": 112, "occupied_max": 106181}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4514, "budget_exhausted": false, "seconds": 156.0, "requests": 20, "tool_calls": 29, "occupied_max": 16235}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1441.7, "requests": 72, "tool_calls": 105, "occupied_max": 107895}, {"instance_id": "github__spec-kit-4559", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 890.8, "requests": 67, "tool_calls": 88, "occupied_max": 109247}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1134.9, "requests": 72, "tool_calls": 80, "occupied_max": 126826}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9105, "budget_exhausted": false, "seconds": 217.7, "requests": 34, "tool_calls": 57, "occupied_max": 29855}, {"instance_id": "dstackai__dstack-4256", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 59639, "budget_exhausted": false, "seconds": 827.2, "requests": 27, "tool_calls": 48, "occupied_max": 72666}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 967.9, "requests": 55, "tool_calls": 84, "occupied_max": 96984}, {"instance_id": "marin-community__marin-9419", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 872.3, "requests": 36, "tool_calls": 57, "occupied_max": 121388}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20876, "budget_exhausted": false, "seconds": 1163.6, "requests": 40, "tool_calls": 53, "occupied_max": 47260}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 896.7, "requests": 31, "tool_calls": 52, "occupied_max": 72330}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 57557, "budget_exhausted": false, "seconds": 1692.2, "requests": 60, "tool_calls": 75, "occupied_max": 88901}, {"instance_id": "ansible__ansible-87624", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 844.8, "requests": 42, "tool_calls": 75, "occupied_max": 108710}, {"instance_id": "scipy__scipy-26156", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 795.2, "requests": 28, "tool_calls": 43, "occupied_max": 78576}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 985.5, "requests": 35, "tool_calls": 52, "occupied_max": 85773}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 776.0, "requests": 31, "tool_calls": 38, "occupied_max": 67419}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 813.7, "requests": 39, "tool_calls": 44, "occupied_max": 75902}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1399.1, "requests": 61, "tool_calls": 84, "occupied_max": 121670}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 971.2, "requests": 52, "tool_calls": 79, "occupied_max": 101253}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 17271, "budget_exhausted": false, "seconds": 392.0, "requests": 26, "tool_calls": 44, "occupied_max": 42651}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1085.5, "requests": 62, "tool_calls": 99, "occupied_max": 111500}, {"instance_id": "roboflow__supervision-2600", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 774.7, "requests": 31, "tool_calls": 41, "occupied_max": 74528}] list
tool_calls3,084 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures14 responses {"wire": {"invalid_tool_call": 5, "unparsed_markup": 3}, "harness_format_errors": {"truncated_no_tool_call": 72, "no_tool_call": 2, "unknown_tool": 5, "missing_command": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations72 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors2 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits14 hits {"ngram64x3": 13, "same_call_x4": 1, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}
decode_whole_run_tps85.2 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit5a73ef5dd71c904cc9d6af503d7a53c401b53a98 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T021336Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T144136Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass needle T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-12 00:41 AEST
Finished2026-10-12 00:44 AEST (2.5 min)
Exit status0
Noteskit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max104,552 tokens
context.headroom_minunavailable no occupancy measured in this run
needle_pass_rate1.00 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 49.5}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 49.0}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 49.2}]
needle_max_passed_tokens104,552 tokens

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit5a73ef5dd71c904cc9d6af503d7a53c401b53a98 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T144136Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2

20261011T144405Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-

✓ pass quality eval T46 eligibility: none

Configstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
Started2026-10-12 00:44 AEST
Finished2026-10-12 05:14 AEST (270.1 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources

Launch

recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefc
MTP / speculativeoff
argv/opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0
envCUDA_DEVICE_ORDER=PCI_BUS_ID
PORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_max94,117 tokens
context.headroom_min36,842 tokens
tasks_attempted30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved21 tasks {"eval_errors": 0}
solve_rate0.7 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17294, "budget_exhausted": false, "seconds": 508.7, "requests": 29, "tool_calls": 38, "occupied_max": 30585}, {"instance_id": "sympy__sympy-20442", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 882.2, "requests": 43, "tool_calls": 60, "occupied_max": 84206}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29915, "budget_exhausted": false, "seconds": 514.5, "requests": 35, "tool_calls": 58, "occupied_max": 49083}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": false, "seconds": 758.7, "requests": 21, "tool_calls": 23, "occupied_max": 57688}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6529, "budget_exhausted": false, "seconds": 130.3, "requests": 19, "tool_calls": 24, "occupied_max": 13813}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31740, "budget_exhausted": false, "seconds": 550.3, "requests": 67, "tool_calls": 90, "occupied_max": 72248}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25802, "budget_exhausted": false, "seconds": 396.0, "requests": 17, "tool_calls": 29, "occupied_max": 34952}, {"instance_id": "sympy__sympy-18199", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 742.1, "requests": 26, "tool_calls": 37, "occupied_max": 60261}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15357, "budget_exhausted": false, "seconds": 257.9, "requests": 24, "tool_calls": 40, "occupied_max": 30358}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11509, "budget_exhausted": false, "seconds": 193.2, "requests": 22, "tool_calls": 29, "occupied_max": 26770}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17337, "budget_exhausted": false, "seconds": 289.6, "requests": 25, "tool_calls": 33, "occupied_max": 34039}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14445, "budget_exhausted": false, "seconds": 280.3, "requests": 37, "tool_calls": 41, "occupied_max": 34051}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10445, "budget_exhausted": false, "seconds": 176.5, "requests": 29, "tool_calls": 29, "occupied_max": 21988}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 50151, "budget_exhausted": false, "seconds": 806.7, "requests": 66, "tool_calls": 103, "occupied_max": 85662}, {"instance_id": "sympy__sympy-12171", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 772.1, "requests": 30, "tool_calls": 45, "occupied_max": 80377}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 825.8, "requests": 61, "tool_calls": 75, "occupied_max": 76415}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27782, "budget_exhausted": false, "seconds": 443.7, "requests": 51, "tool_calls": 55, "occupied_max": 51571}, {"instance_id": "django__django-16873", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 751.6, "requests": 29, "tool_calls": 40, "occupied_max": 63077}, {"instance_id": "astropy__astropy-6938", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 906.0, "requests": 39, "tool_calls": 52, "occupied_max": 76037}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 32715, "budget_exhausted": false, "seconds": 516.7, "requests": 51, "tool_calls": 60, "occupied_max": 57005}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12439, "budget_exhausted": false, "seconds": 220.2, "requests": 19, "tool_calls": 34, "occupied_max": 28899}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28600, "budget_exhausted": false, "seconds": 501.1, "requests": 56, "tool_calls": 78, "occupied_max": 59565}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 795.2, "requests": 38, "tool_calls": 50, "occupied_max": 76143}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40374, "budget_exhausted": false, "seconds": 703.0, "requests": 38, "tool_calls": 56, "occupied_max": 59447}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4831, "budget_exhausted": false, "seconds": 91.5, "requests": 15, "tool_calls": 21, "occupied_max": 10864}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 35357, "budget_exhausted": false, "seconds": 558.6, "requests": 37, "tool_calls": 51, "occupied_max": 64685}, {"instance_id": "django__django-14672", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 802.5, "requests": 33, "tool_calls": 33, "occupied_max": 72991}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22364, "budget_exhausted": false, "seconds": 283.2, "requests": 14, "tool_calls": 20, "occupied_max": 19070}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 56596, "budget_exhausted": false, "seconds": 865.1, "requests": 60, "tool_calls": 78, "occupied_max": 94117}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17585, "budget_exhausted": false, "seconds": 264.5, "requests": 24, "tool_calls": 36, "occupied_max": 31663}] list
tool_calls1,418 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures5 responses {"wire": {"invalid_tool_call": 4}, "harness_format_errors": {"truncated_no_tool_call": 18, "invalid_arguments": 1, "unknown_tool": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations18 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits21 hits {"ngram64x3": 21, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}
decode_whole_run_tps86.0 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit5a73ef5dd71c904cc9d6af503d7a53c401b53a98 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T144405Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb

20261011T191447Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-load

✓ pass load T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb
Started2026-10-12 05:14 AEST
Finished2026-10-12 05:15 AEST (0.6 min)
Exit statusNone
Notes–

Other runs of this config: load lab

Launch

recorded by tools/run.py (artifacts/runs/20261011T191447Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds34.1 s
host_ram_drop_gb55.6 GB
vram_ready_mib23,660 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit3d9a6cab88549c415b7c1800e61a577ac052a277 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T191447Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb

20261011T191522Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-lab

✓ pass lab T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb
Started2026-10-12 05:15 AEST
Finished2026-10-12 05:17 AEST (2.1 min)
Exit status0
Notesofficial lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-iq3-s-128k-rtx-3090-24gb.128k.20261011T191729.json; recipe written only on pass

Other runs of this config: load lab

Launch

recorded by tools/run.py (artifacts/runs/20261011T191447Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fb
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=IQ3_S
GGUF_DIR=/gguf/IQ3_S
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps75.0 tok/s
lab_prefill_tps2,174 tok/s "official lab.py try"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit3d9a6cab88549c415b7c1800e61a577ac052a277 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T191522Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb

20261011T191739Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load

✓ pass load T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
Started2026-10-12 05:17 AEST
Finished2026-10-12 05:18 AEST (0.4 min)
Exit statusNone
Notes–

Other runs of this config: load load lab lab lab lab

Launch

recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds25.0 s
host_ram_drop_gb39.6 GB
vram_ready_mib23,646 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit778868340d6979684600554d81a6f1354d7a3a8d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T191739Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb

20261011T191806Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab

✓ pass lab T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
Started2026-10-12 05:18 AEST
Finished2026-10-12 05:20 AEST (2.7 min)
Exit status0
Notesofficial lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261011T192046.json; recipe written only on pass

Other runs of this config: load load lab lab lab lab

Launch

recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps103 tok/s
lab_prefill_tps2,644 tok/s "official lab.py try"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit778868340d6979684600554d81a6f1354d7a3a8d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T191806Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb

20261011T192047Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab

✓ pass lab T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
Started2026-10-12 05:20 AEST
Finished2026-10-12 05:23 AEST (3.0 min)
Exit status0
Notesofficial lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261011T192346.json; recipe written only on pass

Other runs of this config: load load lab lab lab lab

Launch

recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps101 tok/s
lab_prefill_tps2,631 tok/s "official lab.py try"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit778868340d6979684600554d81a6f1354d7a3a8d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T192047Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb

20261011T192347Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab

✓ pass lab T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
Started2026-10-12 05:23 AEST
Finished2026-10-12 05:26 AEST (3.0 min)
Exit status0
Notesofficial lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261011T192647.json; recipe written only on pass

Other runs of this config: load load lab lab lab lab

Launch

recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps88.2 tok/s
lab_prefill_tps2,638 tok/s "official lab.py try"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit778868340d6979684600554d81a6f1354d7a3a8d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261011T192347Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2

20261011T192654Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-load

✓ pass load T48 eligibility: none

Configsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
Started2026-10-12 05:26 AEST
Finished2026-10-12 05:28 AEST (1.9 min)
Exit statusNone
Notes–

Other runs of this config: load lab lab stability quality eval

Launch

recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Digestsha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cf
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envCUDA_DEVICE_ORDER=PCI_BUS_ID
HF_HUB_OFFLINE=1
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_PREFILL_FP16_ACC=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
SGLANG_EXL3_VIT_SDPA=1
SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1
SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured212,992 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds113 s
host_ram_drop_gb11.1 GB
vram_ready_mib21,270 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commite265bcccc6f30794949bee93596736937e910e5d dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261011T192654Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-load.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2

20261011T192849Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab

✓ pass lab T48 eligibility: none

Configsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
Started2026-10-12 05:28 AEST
Finished2026-10-12 05:33 AEST (5.1 min)
Exit status0
Notesserved=turboderp-Qwen3.8-27B-exl3-3.00bpw

Other runs of this config: load lab lab stability quality eval

Launch

recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Digestsha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cf
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envCUDA_DEVICE_ORDER=PCI_BUS_ID
HF_HUB_OFFLINE=1
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_PREFILL_FP16_ACC=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
SGLANG_EXL3_VIT_SDPA=1
SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1
SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured212,992 tokens
context.occupied_max173,372 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps126 tok/s
lab_prefill_tps831 tok/s "context-gate prompt tokens / total request seconds"
mtp_acceptance_rate0.7354 fraction {"mean_accept_len": 3.207, "batches": 101, "source": "SGLang decode log"}
mtp_accepted_tps126 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261011T192849Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2

20261011T193357Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-stability

✓ pass stability T48 eligibility: none

Configsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
Started2026-10-12 05:33 AEST
Finished2026-10-12 05:44 AEST (10.6 min)
Exit status0
Notesstability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 134 tool calls in this run count toward the config's 100-call minimum (§5)

Other runs of this config: load lab lab stability quality eval

Launch

recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Digestsha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cf
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envCUDA_DEVICE_ORDER=PCI_BUS_ID
HF_HUB_OFFLINE=1
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_PREFILL_FP16_ACC=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
SGLANG_EXL3_VIT_SDPA=1
SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1
SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured212,992 tokens
context.occupied_max68,013 tokens
context.headroom_min144,269 tokens
duration_s600 s {"cap_s": 600.0, "driver_capped": true}
tool_calls134 calls {"per_min": 13.4, "requests": 107}
parse_failures0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations0 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}
decode_window_min_tps90.1 tok/s
window 1: 112.88 tok/swindow 2: 116.11 tok/swindow 3: 112.69 tok/swindow 4: 102.4 tok/swindow 5: 99.84 tok/swindow 6: 116.3 tok/smin 99.84 · max 116.3 tok/s
{"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 333, "min_at_generation_s": 329.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}
decode_whole_run_tps113 tok/s {"generation_seconds": 452.722, "generated_tokens": 51092.2}
tasks_attempted3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}
tasks_solved2 attempts {"eval_errors": 0, "swebench": "5.0.2"}
crash_or_oomno {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}
gates_afterskipped filled by qualify step: the lab run that follows this run (tools/qualify.py)
agent_attempts[{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 52.3, "requests": 16, "tool_calls": 20, "completion_tokens": 3350, "occupied_max": 11234}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 457.9, "requests": 55, "tool_calls": 66, "completion_tokens": 40203, "occupied_max": 68013}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 89.8, "requests": 37, "tool_calls": 49, "completion_tokens": 8143, "occupied_max": 20955}] list
tasks_over_64k["sympy__sympy-21612"] instance ids
completion_tokens51,696 tokens {"finish_reasons": {"tool_calls": 107}}
mtp_acceptance_rate0.6951 fraction {"mean_accept_len": 3.086, "batches": 421, "source": "SGLang decode log"}
mtp_accepted_tps113 tok/s "output tok/s including accepted draft tokens"

Agentic summary: 600 s, 134 tool calls, 0 parse failures, 0 repetition hits, tasks solved 2/3, decode min window 90.1 tok/s, whole run 113 tok/s.

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261011T193357Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-stability.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2

20261011T194435Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab

✓ pass lab T48 eligibility: none

Configsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
Started2026-10-12 05:44 AEST
Finished2026-10-12 05:49 AEST (4.6 min)
Exit status0
Notesserved=turboderp-Qwen3.8-27B-exl3-3.00bpw

Other runs of this config: load lab lab stability quality eval

Launch

recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Digestsha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cf
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envCUDA_DEVICE_ORDER=PCI_BUS_ID
HF_HUB_OFFLINE=1
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_PREFILL_FP16_ACC=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
SGLANG_EXL3_VIT_SDPA=1
SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1
SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured212,992 tokens
context.occupied_max173,372 tokens
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps139 tok/s
lab_prefill_tps836 tok/s "context-gate prompt tokens / total request seconds"
mtp_acceptance_rate0.8036 fraction {"mean_accept_len": 3.413, "batches": 75, "source": "SGLang decode log"}
mtp_accepted_tps139 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261011T194435Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab.json

Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2

20261011T194914Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-quality_eval

✓ pass quality eval T48 eligibility: none

Configsglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
Started2026-10-12 05:49 AEST
Finished2026-10-12 11:47 AEST (358.4 min)
Exit status0
Notesquality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum

Other runs of this config: load lab lab stability quality eval

Launch

recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt)
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
Imageghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Digestsha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79
Enginesglang-exl3
Engine source–
Launch filelaunches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cf
MTP / speculative{"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}
argv/opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt
envCUDA_DEVICE_ORDER=PCI_BUS_ID
HF_HUB_OFFLINE=1
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
SGLANG_EXL3_EMBED_HOST=1
SGLANG_EXL3_PREFILL_FP16_ACC=1
SGLANG_EXL3_VIT_MLP_CHUNK=8192
SGLANG_EXL3_VIT_SDPA=1
SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1
SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3@6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json

Metrics

context.configured212,992 tokens
context.occupied_max99,327 tokens
context.headroom_min109,482 tokens
tasks_attempted50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}
tasks_solved39 tasks {"eval_errors": 1}
solve_rate0.78 fraction
token_budget_per_task60,000 completion tokens
per_task[{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29667, "budget_exhausted": false, "seconds": 508.4, "requests": 61, "tool_calls": 83, "occupied_max": 68863}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 12864, "budget_exhausted": false, "seconds": 260.1, "requests": 28, "tool_calls": 37, "occupied_max": 32478}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22223, "budget_exhausted": false, "seconds": 507.7, "requests": 76, "tool_calls": 83, "occupied_max": 54810}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7718, "budget_exhausted": false, "seconds": 74.2, "requests": 16, "tool_calls": 23, "occupied_max": 17777}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9397, "budget_exhausted": false, "seconds": 193.1, "requests": 32, "tool_calls": 40, "occupied_max": 28500}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 49962, "budget_exhausted": false, "seconds": 641.9, "requests": 56, "tool_calls": 61, "occupied_max": 71289}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17446, "budget_exhausted": false, "seconds": 438.2, "requests": 49, "tool_calls": 56, "occupied_max": 36321}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 32018, "budget_exhausted": false, "seconds": 358.4, "requests": 39, "tool_calls": 49, "occupied_max": 49496}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19250, "budget_exhausted": false, "seconds": 220.9, "requests": 34, "tool_calls": 48, "occupied_max": 43629}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15718, "budget_exhausted": false, "seconds": 177.1, "requests": 26, "tool_calls": 34, "occupied_max": 39389}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12790, "budget_exhausted": false, "seconds": 169.4, "requests": 21, "tool_calls": 22, "occupied_max": 22905}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 44893, "budget_exhausted": false, "seconds": 553.5, "requests": 53, "tool_calls": 60, "occupied_max": 68495}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16488, "budget_exhausted": false, "seconds": 214.2, "requests": 22, "tool_calls": 32, "occupied_max": 30899}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13996, "budget_exhausted": false, "seconds": 226.0, "requests": 36, "tool_calls": 43, "occupied_max": 29618}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 12596, "budget_exhausted": false, "seconds": 169.1, "requests": 28, "tool_calls": 33, "occupied_max": 31333}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18143, "budget_exhausted": false, "seconds": 258.2, "requests": 17, "tool_calls": 21, "occupied_max": 19196}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14547, "budget_exhausted": false, "seconds": 331.5, "requests": 36, "tool_calls": 48, "occupied_max": 36517}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6212, "budget_exhausted": false, "seconds": 105.4, "requests": 19, "tool_calls": 25, "occupied_max": 19479}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 42365, "budget_exhausted": false, "seconds": 461.9, "requests": 39, "tool_calls": 53, "occupied_max": 62092}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12636, "budget_exhausted": false, "seconds": 125.4, "requests": 18, "tool_calls": 28, "occupied_max": 25241}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 55326, "budget_exhausted": false, "seconds": 849.2, "requests": 62, "tool_calls": 68, "occupied_max": 90422}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37313, "budget_exhausted": false, "seconds": 425.6, "requests": 42, "tool_calls": 56, "occupied_max": 39298}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25453, "budget_exhausted": false, "seconds": 230.8, "requests": 36, "tool_calls": 47, "occupied_max": 31915}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33850, "budget_exhausted": false, "seconds": 343.0, "requests": 34, "tool_calls": 43, "occupied_max": 54816}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10138, "budget_exhausted": false, "seconds": 149.1, "requests": 29, "tool_calls": 37, "occupied_max": 24960}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22085, "budget_exhausted": false, "seconds": 312.5, "requests": 39, "tool_calls": 59, "occupied_max": 50513}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 641.4, "requests": 34, "tool_calls": 42, "occupied_max": 54175}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 38476, "budget_exhausted": false, "seconds": 452.1, "requests": 50, "tool_calls": 72, "occupied_max": 64416}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 52535, "budget_exhausted": false, "seconds": 612.0, "requests": 87, "tool_calls": 109, "occupied_max": 83204}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4988, "budget_exhausted": false, "seconds": 190.4, "requests": 21, "tool_calls": 27, "occupied_max": 16013}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 23809, "budget_exhausted": false, "seconds": 1257.9, "requests": 109, "tool_calls": 127, "occupied_max": 62642}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28165, "budget_exhausted": false, "seconds": 540.4, "requests": 60, "tool_calls": 83, "occupied_max": 67025}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 57625, "budget_exhausted": false, "seconds": 644.5, "requests": 42, "tool_calls": 38, "occupied_max": 54696}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8598, "budget_exhausted": false, "seconds": 194.9, "requests": 40, "tool_calls": 50, "occupied_max": 25888}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13442, "budget_exhausted": false, "seconds": 197.9, "requests": 41, "tool_calls": 53, "occupied_max": 36955}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25221, "budget_exhausted": false, "seconds": 322.6, "requests": 42, "tool_calls": 53, "occupied_max": 55458}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 45787, "budget_exhausted": false, "seconds": 510.6, "requests": 36, "tool_calls": 44, "occupied_max": 72796}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19860, "budget_exhausted": false, "seconds": 1117.5, "requests": 74, "tool_calls": 85, "occupied_max": 55676}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8397, "budget_exhausted": false, "seconds": 153.6, "requests": 15, "tool_calls": 21, "occupied_max": 17233}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16622, "budget_exhausted": false, "seconds": 230.5, "requests": 24, "tool_calls": 30, "occupied_max": 39013}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 41882, "budget_exhausted": false, "seconds": 491.0, "requests": 83, "tool_calls": 114, "occupied_max": 77462}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26034, "budget_exhausted": false, "seconds": 465.9, "requests": 47, "tool_calls": 60, "occupied_max": 41930}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9951, "budget_exhausted": false, "seconds": 252.9, "requests": 32, "tool_calls": 50, "occupied_max": 26069}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16914, "budget_exhausted": false, "seconds": 164.0, "requests": 22, "tool_calls": 27, "occupied_max": 28474}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34673, "budget_exhausted": false, "seconds": 790.4, "requests": 61, "tool_calls": 68, "occupied_max": 63418}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 805.0, "requests": 54, "tool_calls": 60, "occupied_max": 63073}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 36398, "budget_exhausted": false, "seconds": 464.3, "requests": 49, "tool_calls": 65, "occupied_max": 59467}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 11499, "budget_exhausted": false, "seconds": 135.0, "requests": 21, "tool_calls": 31, "occupied_max": 27207}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 811.1, "requests": 61, "tool_calls": 82, "occupied_max": 99327}, {"instance_id": "roboflow__supervision-2600", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 538.5, "requests": 20, "tool_calls": 18, "occupied_max": 33907}] list
tool_calls2,598 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}
parse_failures1 responses {"wire": {"invalid_tool_call": 1}, "harness_format_errors": {"truncated_no_tool_call": 27}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}
output_truncations27 responses {"rule": "cut off at max_tokens with no tool call"}
agent_format_errors0 responses {"rule": "agent protocol errors other than malformed tool calls"}
repetition_hits43 hits {"ngram64x3": 43, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}
decode_whole_run_tps112 tok/s
kldunavailable not produced by the agentic evaluation; see quality_panel / KLD records
top1_agreementunavailable not produced by the agentic evaluation; see quality_panel records
mtp_acceptance_rate0.6948 fraction {"mean_accept_len": 3.085, "batches": 10995, "source": "SGLang decode log"}
mtp_accepted_tps112 tok/s "output tok/s including accepted draft tokens"

Cache, layout and host

Cachewarm
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-27b/20261011T194914Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-quality_eval.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb

20261012T014808Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load

✓ pass load T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
Started2026-10-12 11:48 AEST
Finished2026-10-12 11:48 AEST (0.4 min)
Exit statusNone
Notes–

Other runs of this config: load load lab lab lab lab

Launch

recorded by tools/run.py (artifacts/runs/20261012T014808Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
load_seconds25.0 s
host_ram_drop_gb39.4 GB
vram_ready_mib23,646 MiB

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit47c9862242d864e21d266cc77304505cf845d531 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261012T014808Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load.json

Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb

20261012T014835Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab

✓ pass lab T47 eligibility: none

Configstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
Started2026-10-12 11:48 AEST
Finished2026-10-12 11:50 AEST (2.0 min)
Exit status0
Notesofficial lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261012T015035.json; recipe written only on pass

Other runs of this config: load load lab lab lab lab

Launch

recorded by tools/run.py (artifacts/runs/20261012T014808Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt)
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Imagesha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172
Digest–
Enginestrata
Engine sourcehttps://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c
Launch filelaunches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0f
MTP / speculativeoff
argv
envFAMILY=qwen
MODEL=Q2_0
GGUF_DIR=/gguf/Q2_0
CONTEXT=131072
VISION=no
KV=int8
GPU=0
LOW_RAM=off
PORT=8080
CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json

Metrics

context.configured131,072 tokens
context.occupied_maxunavailable no occupancy measured in this run
context.headroom_minunavailable no occupancy measured in this run
gates_passed["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}
lab_decode_c1_tps99.7 tok/s
lab_prefill_tps2,664 tok/s "official lab.py try"
mtp_accepted_tpsskipped speculative decoding off in this launch
mtp_acceptance_rateskipped speculative decoding off in this launch

Cache, layout and host

Cachecold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache
Layoutcurrent · 1 card(s) used · display on
hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
cpus_offline2,18
cpu_boostFalse
cpu_max_mhz3401
machine_checks_this_boot6
#UsedNameUUIDBusPCIeDisplay
0NVIDIA GeForce RTX 3090GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
1NVIDIA GeForce RTX 3090GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
2●NVIDIA GeForce RTX 3090GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

Provenance

bench commit47c9862242d864e21d266cc77304505cf845d531 dirty tree
pins.json sha256858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)
registryd21258dd744e7c78be28177c90060c6af8e10b7e
kitef883d269e50ecbea290f919f095e2f3ca633b42
mini-swe-agent04d809ceab9df28f9adaed044884180159172930

Artifacts and raw output

Raw record: results/qwen3.8-flash-next/20261012T014835Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json

Moved. Results: Findings › Coding quality. Construction: Method & Hardware.

Superseded. This page is kept for old links; the coding results below are memorised; see fresh-issues results.

Qwen3.8-Flash-Next vs Qwen3.8-27B

Exploratory deployment comparison on this box (decision 0014): each model as best deployed on one RTX 3090 at PCIe 4.0 x8 (GPU2), same harness, task set, sampling and token budget. Flash-Next: PR #144 launch on a21efea6 with the stability fix (chunked prefill 8192). 27B: the operator's daily server (EXL3 3.00 bpw, NEXTN/MTP, single stream). Conclusions are per priority, not "better overall".

Coding quality on this page is memorised. The SWE-bench-Lite tasks below are public; Flash-Next reproduces the original fixes almost word for word. The contamination-reduced comparison is on the Fresh-issues benchmark page.

MetricFlash-Next27B
Agentic coding — 30 SWE-bench-Lite tasks, one uncapped attempt each
Tasks solved3018
Solve rate100.0%60.0%
Tool calls1,4451,453
Parse failures15
Repetition hits1672
Decode during the run42.3 tok/s125.5 tok/s
10-minute agentic run (8-task §5 set)
Tool calls69137
Slowest 60-s window39.8 tok/s111.1 tok/s
Whole-run decode43.6 tok/s119.7 tok/s
Repetition hits01
Speed (kit protocol, early exit off; lab gates)
Decode, 1 request47.5 tok/s120.9 tok/s
Decode, 1 request at 32k48.5 tok/s111.6 tok/s
Decode, 4 requests (aggregate)54.3 tok/s122.2 tok/s
Prefill 8k2,285 tok/s1,076 tok/s
Prefill 64k2,431 tok/s854 tok/s
TTFT prefill 8k2,284 tok/s1,034 tok/s
Lab decode C148.8 tok/s143.7 tok/s
Lab prefill2,131 tok/s740 tok/s
MTP acceptance—56.3%
Context and footprint
Real KV capacity (server log)209,984 tokens135,424 tokens
Needle pass rate100.0%75.0%
Largest needle passed191,045 tokens104,552 tokens
Load time260 s127 s
Host RAM used65.6 GB7.5 GB
VRAM at ready20,844 MiB22,038 MiB

Contamination check

SWE-bench-Lite fixes are public since 2023, so a model may have seen them in training. For each task we measured the share of the real upstream fix's added lines that the model's patch reproduces verbatim. A model that solves by recall reproduces the gold patch; one that solves by reasoning writes its own.

ModelMedian share of the gold fix reproducedTasks with ≥ 90 % reproduced
Flash-Next100 %24 of 30
27B14 %4 of 27

Reading: a high share means the solve rate largely measures memorisation of these public fixes, not agentic problem-solving. The solve rates below are reported as measured, with this caveat; a decontaminated task set (issues newer than both models) is the fair test.

Paired outcome on the same 30 tasks

Flash-Next only: 12 · 27B only: 0 · both or neither: 18. Solve-rate difference (Flash-Next − 27B) +40.0 pp (95 % CI +22.5 to +57.5, Wald); exact McNemar p = 0.000488.

TaskFlash-Nexttokensmin27Btokensmin
matplotlib__matplotlib-26011✓ solved · 86 % gold24,48721.4✓ solved · 14 % gold21,3955.0
sympy__sympy-20442✓ solved · 20 % gold14,32110.8✓ solved · 20 % gold19,9923.5
psf__requests-2148✓ solved · 100 % gold11,5698.7✓ solved · 100 % gold54,0568.9
scikit-learn__scikit-learn-25570✓ solved · 0 % gold20,39912.9✓ solved · 0 % gold33,0305.8
django__django-14752✓ solved · 22 % gold9,3705.6✓ solved · 33 % gold9,9091.7
django__django-13158✓ solved · 100 % gold21,29814.9✓ solved · 0 % gold27,5764.7
sympy__sympy-21171✓ solved · 100 % gold9,6226.2✗ · 33 % gold4,9961.9
sympy__sympy-18199✓ solved · 96 % gold38,14523.1✗ · 2 % gold16,3782.9
django__django-11049✓ solved · 100 % gold9,8535.4✓ solved · 100 % gold10,8251.8
django__django-13028✓ solved · 100 % gold5,8443.6✓ solved · 0 % gold29,9075.5
django__django-13551✓ solved · 100 % gold12,9397.9✗ · 0 % gold28,7794.2
pallets__flask-4045✓ solved · 100 % gold10,1586.3✗ · 17 % gold14,3382.8
sympy__sympy-19487✓ solved · 100 % gold9,24224.2✗ · 50 % gold11,4562.2
mwaskom__seaborn-3190✓ solved · 100 % gold12,53214.6✗ · 0 % gold50,91311.0
sympy__sympy-12171✓ solved · 100 % gold18,7409.5✗60,0008.4
sphinx-doc__sphinx-7738✓ solved · 100 % gold11,5286.6✓ solved · 100 % gold48,3199.1
scikit-learn__scikit-learn-13497✓ solved · 100 % gold15,2587.3✓ solved · 12 % gold7,9932.0
django__django-16873✓ solved · 33 % gold11,4386.0✓ solved · 33 % gold11,2801.9
astropy__astropy-6938✓ solved · 100 % gold27,19315.2✓ solved · 0 % gold14,5243.9
django__django-14667✓ solved · 100 % gold11,4268.1✗ · 0 % gold10,7642.0
scikit-learn__scikit-learn-11040✓ solved · 100 % gold10,8735.8✗ · 18 % gold19,3493.5
sphinx-doc__sphinx-10325✓ solved · 100 % gold18,64213.4✓ solved · 14 % gold36,6836.3
scikit-learn__scikit-learn-11281✓ solved · 96 % gold22,23312.9✓ solved · 48 % gold25,1934.0
psf__requests-1963✓ solved · 100 % gold11,1569.2✓ solved · 0 % gold34,8576.4
scikit-learn__scikit-learn-13439✓ solved · 40 % gold9,1485.4✓ solved · 40 % gold2,8150.5
matplotlib__matplotlib-25498✓ solved · 100 % gold26,21518.9✗60,0008.0
django__django-14672✓ solved · 100 % gold6,0923.5✓ solved · 100 % gold6,6281.3
django__django-11964✓ solved · 100 % gold15,0497.6✗ · 0 % gold54,1248.6
django__django-16408✓ solved · 100 % gold20,16812.3✗60,0009.6
django__django-14787✓ solved · 100 % gold10,4876.5✓ solved · 0 % gold24,9504.2

Hardware & methodology

Host-to-device bandwidth

RunOutcomeLayoutGPUBusPCIeH2DD2HNotes
2026-10-03 06:53 AEST✓ passcurrentGPU2 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x813.5 GB/s13.2 GB/s1 GiB pinned transfers; full table in detail
2026-10-03 06:53 AEST✓ passcurrentGPU0 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x86.03 GB/s6.60 GB/s1 GiB pinned transfers; full table in detail
2026-10-04 19:11 AEST✓ passcurrentGPU1 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x813.5 GB/s13.2 GB/s1 GiB pinned transfers; full table in detail

NVMe random reads

RunOutcomeIOPSThroughputDetailNotes
2026-10-03 06:54 AEST✓ pass168,600 IOPS3.35 GB/s"16 KiB, 32 threads"~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5/ngram_embedding.safetensors

Host

hostnameomarchy-gpu
cpuAMD Ryzen 9 5950X 16-Core Processor
ram_total_gib125.7
ram_layoutDIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels)
kernel7.2.5-3-omarchy
driver610.57.04
cuda13.3
cpus_online0-1,3-17,19-31
0-31 (changed between runs)
cpus_offline2,18
none (changed between runs)
machine_checks_this_boot0
6 (changed between runs)
cpu_boostFalse
cpu_max_mhz3401
LayoutGPU UUIDBusPCIeDisplay
currentGPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8off
currentGPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8on
currentGPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8off

GPUs as seen by the records (PCIe width is the link width at record time).

Dropped configs and why

ModelConfigFailed runsWhy (record notes)
GLMexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2! fail lab / gates ! fail TTFT / invalid measurement ! fail stability ! fail lab / gatesserved=glm-5.3-flash
INVALIDATED: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2. headline length 8192; prompts carry random nonces
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 21 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 3.356 < floor 15.0
served=glm-5.3-flash
GLMglm-llamacpp-iq2m-128k-1x-gpu2! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.91 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2! fail lab / gates ! fail kit sweep / timeoutserved=glm-5.3-flash
(no notes)
GLMglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2! fail lab / gatesserved=glm-5.3-flash
GLMik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 47 tool-call parse failure(s); decode window min 9.458 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2! fail lab / gates ✕ crash stability / measureserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)
GLMik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2! fail lab / gates ✕ crash stability / measureserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)
GLMik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 68 tool-call parse failure(s); decode window min 13.255 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012✕ crash load / loadche_init: CUDA2 KV buffer size = 570.81 MiB llama_init_from_model: KV self size = 748.00 MiB, c^KV (q8_0): 748.00 MiB, kv^T: not used llama_init_from_model: KV self indexer size = 704.00 MiB (f16) llama_init_from_model: CUDA_Host output buffer size = 0.59 MiB ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696 llama_init_from_model: failed to allocate compute buffers ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'
GLMik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012✕ crash load / load c^KV (q8_0): 68.00 MiB, kv^T: not used llama_init_from_model: KV self indexer size = 64.00 MiB (f16) llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2754.25 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 2888039424 llama_init_from_model: failed to allocate compute buffers ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs common_speculative_init: failed to create MTP context srv init: failed to initialize recurrent speculative context ~ggml_backend_cuda_context: have 13 graphs ~ggml_backend_cuda_context: have 14 graphs ~ggml_backend_cuda_context: have 11 graphs
GLMik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012! fail lab / gates ✕ crash stability / measureserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)
GLMik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012✕ crash load / loadche_init: CUDA2 KV buffer size = 570.81 MiB llama_init_from_model: KV self size = 816.00 MiB, c^KV (q8_0): 816.00 MiB, kv^T: not used llama_init_from_model: KV self indexer size = 768.00 MiB (f16) llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696 llama_init_from_model: failed to allocate compute buffers ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs ~ggml_backend_cuda_context: have 0 graphs llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 30 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.854 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 27 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 8.847 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.035 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012! fail lab / gates ! fail stability ✕ crash lab / harnessserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 26 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.388 < floor 15.0
(no notes)
GLMllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.646 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 22 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 9.855 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
GLMllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.807 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf
GLMllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf
stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 19 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.8 < floor 15.0
served=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf
27Bsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2! fail lab / gates ! fail needle / measure ! fail needle / measure(no notes)
kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
27Bsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2! fail lab / gates ! fail needle / measure ! fail stability ! fail lab / gates(no notes)
kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 137 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s)
(no notes)
Flash-Nextfn-llamacpp-iq4xs-128k-1x-fit-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 49 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 23.83 < floor 50.0
Flash-Nextfn-llamacpp-iq4xs-128k-1x-gpu2✕ crash load / loaduser, abort 0.18.994.031 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 65358.17 MiB on device 0: cudaMalloc failed: out of memory 0.18.994.037 E alloc_tensor_range: failed to allocate CUDA0 buffer of size 68533006336 0.19.139.695 E llama_model_load: error loading model: unable to allocate CUDA0 buffer 0.19.139.702 E llama_model_load_from_file_impl: failed to load model 0.19.139.708 E cmn common_init_: failed to load model '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf' 0.19.139.713 E srv load_model: failed to load model, '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf' 0.19.139.715 I srv operator(): operator(): cleaning up before exit... 0.19.140.570 E srv llama_server: exiting due to model loading error
Flash-Nextfn-trellis-3.05-0xsero-main-x8-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 63 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.077 < floor 50.0
Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 70 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.626 < floor 50.0
Flash-Nextllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode floor not verifiable (too little generation); agent driver exited 1
Flash-Nextsglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2✕ crash load / load_glue/offload_moe_method.py", line 86, in get_runtime store = HostExpertStore(model_dir, layers, num_experts, hidden, inter, bits, prefix=prefix) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/trellis-serve/cuda/src/sglang_exl3/kernels/offload_store.py", line 62, in __init__ raise NotImplementedError("HostExpertStore: fast relayout implemented for K=3 experts only") NotImplementedError: HostExpertStore: fast relayout implemented for K=3 experts only [2026-10-04 10:55:46] Received sigquit from a child process. It usually means the child failed. [2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=True, pid=1 [2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2! fail stability ! fail stability ! fail stability ! fail needle / measure ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 64 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.539 < floor 50.0
stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 62 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.784 < floor 50.0
stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 61 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.37 < floor 50.0
kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 69 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.78 < floor 50.0
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 53 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.954 < floor 50.0
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 57 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.08 < floor 50.0
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2✕ crash stability / measure ! fail stability ✕ crash stability / measurestability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation)
stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.676 < floor 50.0
stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 6 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation)
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 72 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.915 < floor 50.0
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2✕ crash lab / harness ✕ crash TTFT / measure(no notes)
(no notes)
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.429 < floor 50.0
Flash-Nextsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 67 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.939 < floor 50.0
Flash-Nextsglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2✕ crash load / load ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 116, in __init__ t = parse_table(model_path) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 94, in parse_table raise ValueError(f"{path}: shard_N.trellis tensors missing or not contiguous") ValueError: /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6/ngram_embedding.safetensors: shard_N.trellis tensors missing or not contiguous [2026-10-04 10:56:54] Received sigquit from a child process. It usually means the child failed. [2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=True, pid=1 [2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Flash-Nextstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2! fail kit sweep / measuresweep status: None
Flash-Nextstrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 50 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 2 repetition hit(s)

All failed and crashed runs (84)

StartedOutcomeModelKindConfigKey metricsStageTicket
2026-10-11 11:56 AEST! failFlash-Nextstabilitystrata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2decode_window_min_tps 86.0 tok/s · tool_calls 50 callsT46
2026-10-08 22:55 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2decode_window_min_tps 39.0 tok/s · tool_calls 53 callsT42
2026-10-08 20:38 AEST! failGLMlabglm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2lab_decode_c1_tps 11.3 tok/s · lab_prefill_tps 630 tok/sgatesT42
2026-10-08 17:34 AEST! failGLMkit sweepglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2decode_c1_tps unavailable · prefill_32k_tps unavailabletimeoutT42
2026-10-08 16:34 AEST! failGLMlabglm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2lab_decode_c1_tps 6.10 tok/s · lab_prefill_tps 314 tok/sgatesT42
2026-10-08 16:17 AEST! failFlash-Nextkit sweepstrata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2decode_c1_tps unavailable · prefill_32k_tps unavailablemeasureT41
2026-10-06 03:41 AEST! fail27Bneedlesglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2measureT40
2026-10-06 03:18 AEST! fail27Bneedlesglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2measureT40
2026-10-06 03:15 AEST! fail27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailablegatesT40
2026-10-05 21:35 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 39.8 tok/s · tool_calls 69 callsT40
2026-10-05 21:32 AEST! failFlash-Nextneedlesglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2measureT40
2026-10-05 18:35 AEST! fail27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailablegatesT40
2026-10-05 18:25 AEST! fail27Bstabilitysglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2decode_window_min_tps 111 tok/s · tool_calls 137 callsT40
2026-10-05 18:25 AEST! fail27Bneedlesglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2measureT40
2026-10-05 18:07 AEST! fail27Blabsglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailablegatesT40
2026-10-05 12:13 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 40.4 tok/s · tool_calls 61 callsT14a
2026-10-05 11:27 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 38.8 tok/s · tool_calls 62 callsT14a
2026-10-05 10:49 AEST✕ crashFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_window_min_tps unavailable · tool_calls 6 callsmeasureT13
2026-10-05 10:07 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2decode_window_min_tps 39.5 tok/s · tool_calls 64 callsT14a
2026-10-05 09:22 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2decode_window_min_tps 37.4 tok/s · tool_calls 71 callsT14a
2026-10-04 23:57 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2decode_window_min_tps 40.9 tok/s · tool_calls 72 callsT14b
2026-10-04 23:33 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2decode_window_min_tps 40.1 tok/s · tool_calls 57 callsT14a
2026-10-04 23:02 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_window_min_tps 39.7 tok/s · tool_calls 71 callsT14a
2026-10-04 22:48 AEST✕ crashFlash-NextTTFTsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2ttft_prefill_tps unavailablemeasureT14a
2026-10-04 22:47 AEST✕ crashFlash-Nextlabsglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2lab_decode_c1_tps unavailable · lab_prefill_tps unavailableharnessT14a
2026-10-04 22:32 AEST✕ crashFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2decode_window_min_tps unavailable · tool_calls 0 callsmeasureT13
2026-10-04 21:27 AEST! failGLMlabexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab_decode_c1_tps 7.50 tok/s · lab_prefill_tps 720 tok/sgatesT23
2026-10-04 21:17 AEST! failGLMstabilityexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2decode_window_min_tps 3.36 tok/s · tool_calls 21 callsT23
2026-10-04 21:13 AEST! failGLMTTFTexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2ttft_prefill_tps unavailableinvalid measurementT23
2026-10-04 21:02 AEST! failGLMlabexllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2lab_decode_c1_tps 7.40 tok/s · lab_prefill_tps 739 tok/sgatesT23
2026-10-04 20:56 AEST✕ crashFlash-Nextloadsglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2load_seconds unavailableloadT15
2026-10-04 20:52 AEST✕ crashFlash-Nextloadsglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2load_seconds unavailableloadT15
2026-10-04 20:36 AEST! failFlash-Nextstabilitysglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2decode_window_min_tps 37.9 tok/s · tool_calls 67 callsT13
2026-10-04 19:38 AEST✕ crashGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012decode_window_min_tps unavailable · tool_calls 0 callsmeasureT22c
2026-10-04 19:12 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012lab_decode_c1_tps 10.2 tok/s · lab_prefill_tps 166 tok/sgatesT22c
2026-10-04 17:42 AEST✕ crashGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012load_seconds unavailableloadT22c
2026-10-04 17:31 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab_decode_c1_tps 13.4 tok/s · lab_prefill_tps 189 tok/sgatesT22c
2026-10-04 17:21 AEST! failGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012decode_window_min_tps 13.3 tok/s · tool_calls 0 callsT22c
2026-10-04 16:58 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012lab_decode_c1_tps 14.0 tok/s · lab_prefill_tps 185 tok/sgatesT22c
2026-10-04 16:30 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012lab_decode_c1_tps 11.2 tok/s · lab_prefill_tps 258 tok/sgatesT21
2026-10-04 16:14 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2lab_decode_c1_tps 9.10 tok/s · lab_prefill_tps 370 tok/sgatesT20b
2026-10-04 15:49 AEST! failFlash-Nextstabilityllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012decode_window_min_tps 38.6 tok/s · tool_calls 70 callsT11
2026-10-04 15:19 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 8.30 tok/s · lab_prefill_tps 156 tok/sgatesT21
2026-10-04 15:08 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012decode_window_min_tps 7.81 tok/s · tool_calls 25 callsT21
2026-10-04 14:26 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 8.00 tok/s · lab_prefill_tps 139 tok/sgatesT21
2026-10-04 14:24 AEST✕ crashGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012load_seconds unavailableloadT22c
2026-10-04 14:23 AEST✕ crashGLMloadik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012load_seconds unavailableloadT22c
2026-10-04 14:14 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 288 tok/sgatesT20b
2026-10-04 14:04 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2decode_window_min_tps 8.85 tok/s · tool_calls 27 callsT20b
2026-10-04 13:42 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 288 tok/sgatesT20b
2026-10-04 13:06 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 201 tok/sgatesT20b
2026-10-04 12:49 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 11.8 tok/s · lab_prefill_tps 218 tok/sgatesT21
2026-10-04 12:39 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012decode_window_min_tps 11.0 tok/s · tool_calls 24 callsT21
2026-10-04 12:16 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012lab_decode_c1_tps 11.8 tok/s · lab_prefill_tps 219 tok/sgatesT21
2026-10-04 11:39 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012lab_decode_c1_tps 12.2 tok/s · lab_prefill_tps 153 tok/sgatesT21
2026-10-04 10:42 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab_decode_c1_tps 10.4 tok/s · lab_prefill_tps 132 tok/sgatesT21
2026-10-04 10:31 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012decode_window_min_tps 9.86 tok/s · tool_calls 22 callsT21
2026-10-04 09:30 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012lab_decode_c1_tps 10.5 tok/s · lab_prefill_tps 126 tok/sgatesT21
2026-10-04 09:17 AEST✕ crashGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012lab_decode_c1_tps unavailable · lab_prefill_tps unavailableharnessT21
2026-10-04 09:07 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012decode_window_min_tps 11.4 tok/s · tool_calls 26 callsT21
2026-10-04 08:27 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012lab_decode_c1_tps 12.3 tok/s · lab_prefill_tps 158 tok/sgatesT21
2026-10-04 08:17 AEST! failFlash-Nextstabilityllama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012decode_window_min_tps unavailable · tool_calls 0 callsT11
2026-10-04 06:02 AEST✕ crashGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2decode_window_min_tps unavailable · tool_calls 0 callsmeasureT22b
2026-10-04 05:33 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2lab_decode_c1_tps 8.60 tok/s · lab_prefill_tps 147 tok/sgatesT22b
2026-10-04 05:16 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 199 tok/sgatesT20b
2026-10-04 05:06 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2decode_window_min_tps 8.85 tok/s · tool_calls 30 callsT20b
2026-10-04 04:26 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 200 tok/sgatesT20b
2026-10-04 03:40 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab_decode_c1_tps 8.40 tok/s · lab_prefill_tps 71 tok/sgatesT20b
2026-10-04 03:30 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2decode_window_min_tps 7.65 tok/s · tool_calls 25 callsT20b
2026-10-04 02:26 AEST! failGLMlabllama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2lab_decode_c1_tps 8.30 tok/s · lab_prefill_tps 65 tok/sgatesT20b
2026-10-04 01:51 AEST✕ crashGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2decode_window_min_tps unavailable · tool_calls 0 callsmeasureT22b
2026-10-04 01:19 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2lab_decode_c1_tps 8.20 tok/s · lab_prefill_tps 147 tok/sgatesT22b
2026-10-04 01:01 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 151 tok/sgatesT22b
2026-10-04 00:51 AEST! failGLMstabilityik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2decode_window_min_tps 9.46 tok/s · tool_calls 0 callsT22b
2026-10-04 00:22 AEST! failGLMlabik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2lab_decode_c1_tps 10.0 tok/s · lab_prefill_tps 151 tok/sgatesT22b
2026-10-03 23:31 AEST! failGLMlabllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab_decode_c1_tps 9.30 tok/s · lab_prefill_tps 80 tok/sgatesT20b
2026-10-03 23:21 AEST! failGLMstabilityllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2decode_window_min_tps 8.80 tok/s · tool_calls 19 callsT20b
2026-10-03 21:59 AEST! failGLMlabllama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2lab_decode_c1_tps 9.20 tok/s · lab_prefill_tps 78 tok/sgatesT20b
2026-10-03 12:48 AEST! failGLMlabglm-llamacpp-iq2m-128k-1x-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 81 tok/sgatesT20b
2026-10-03 12:38 AEST! failGLMstabilityglm-llamacpp-iq2m-128k-1x-gpu2decode_window_min_tps 8.91 tok/s · tool_calls 24 callsT20b
2026-10-03 11:41 AEST! failGLMlabglm-llamacpp-iq2m-128k-1x-gpu2lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 81 tok/sgatesT20b
2026-10-03 11:03 AEST! failFlash-Nextstabilityfn-llamacpp-iq4xs-128k-1x-fit-gpu2decode_window_min_tps 23.8 tok/s · tool_calls 49 callsT11
2026-10-03 10:28 AEST✕ crashFlash-Nextloadfn-llamacpp-iq4xs-128k-1x-gpu2load_seconds unavailableloadT11
2026-10-03 08:06 AEST! failFlash-Nextstabilityfn-trellis-3.05-0xsero-main-x8-gpu2decode_window_min_tps 38.1 tok/s · tool_calls 63 callsT10

Decisions

0001-flashnext-llamacpp-quant.md 0001 — Flash-Next llama.cpp comparison quant (T11)

0001 — Flash-Next llama.cpp comparison quant (T11)

Decision: use bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f IQ4_XS (97.7 GB) for both the 1-card and the 3-card llama.cpp comparison, on the attested ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312 image. N-gram table (per_layer_token_embd.weight) pinned to CPU with -ot; experts on CPU for the layers that do not fit (--n-cpu-moe N, tuned once per layout), fp16→q8_0 KV only if needed for 128k+.

Why: closest to 4-bit that fits 24 GB VRAM + ~110 GiB RAM on one card (IQ4_XS ≈ 91 GiB); same file serves the 3-card run, so the 1 vs 3 comparison changes only the layout. Q4_K_M (119.6 GB ≈ 111 GiB) would not leave room on 1 card. No MTP (llama.cpp Flash-Next MTP PR #27836 not merged). Not REAP-pruned.

Limits: the kit quality panel speaks the SGLang API only, so this route gets no kit band → it cannot be the Flash-Next headline (SPEC §5); it is a comparison point. Delete after T11 unless it beats trellis.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • IQ4_XS stays the paired 1 vs 3 card file. Q4_K_M only as a single feasibility screen if IQ4_XS is competitive with trellis (disk is the binding constraint).
  • The 1 vs 3 pair freezes occupied context, KV format, batching and workload; --n-cpu-moe / tensor split per layout are reported, and any KV change is a separate comparison.
  • Full digest + shard hashes via tools/models.py; tensor placement of per_layer_token_embd.weight verified from the server log.
  • Eligibility experiment-only (no kit band). Results kept regardless; weights deleted only after manifests exist.
0001-flashnext-llamacpp-quant.review.md 0001-flashnext-llamacpp-quant.review
  • AMEND. Keep IQ4_XS as the primary paired comparison, but replace the categorical Q4_K_M rejection with a memory-budget check. The strongest counter-argument is that the decision compares the entire file against host RAM, despite offloading weights to VRAM.
  • Q4_K_M is not demonstrably excluded. 119.6 GB ≈ 111.4 GiB. If 18–20 GiB of weights reside on the GPU and their host pages are reclaimable, host weight residency becomes roughly 91–93 GiB, leaving 17–19 GiB within the stated 110 GiB RAM budget. Loading peaks, pinned allocations, KV and scratch could consume that margin; the supplied numbers do not prove they will.
  • Concrete alternative: use IQ4_XS for T11, with one bounded Q4_K_M feasibility screen at the intended context if the allocation budget supports it. Keep any Q4 results as a separate quant comparison. Do not expand into another tuning grid.
  • Fitting is not evidence of adequate speed. Dual-channel DDR4-3200 has a theoretical 51.2 GB/s ceiling. At 50 tok/s, that permits only 1.024 GB of DRAM traffic per token. If all 6B active LM parameters required fresh ideal 4-bit reads, that is ~3 GB/token and at most 17 tok/s before overhead. Record CPU offload placement and measured decode; this is a conditional bandwidth screen, not a performance prediction.
  • “Only the layout changes” needs qualification. Independently tuning --n-cpu-moe N compares each layout’s selected configuration, rather than isolating card-count scaling. Freeze occupied context, KV format, batching and workload across the pair; report both CPU-expert allocations and tensor splits. Treat any KV-format switch as a separate comparison.
  • Complete provenance before launch. Pin the full image digest already supplied in SPEC §6, not just server-cuda12-b11312. Add the exact GGUF shard list and hashes, immutable launches, and the actual tensor-placement evidence for per_layer_token_embd.weight. The proposed -ot setting remains unverified in this review.
  • The quality limitation is correctly identified, but applies to registry eligibility too. Without the required kit band, this Flash-Next configuration is experiment-only, even if all registry lab gates pass. Neither IQ4_XS nor a speed win establishes quality equivalence.
  • Replace “delete unless it beats trellis.” Define the comparison at matched occupied context and cache state, and retain results regardless of outcome. Delete rejected weights only after hashes and immutable references are recorded; retain any finalist under §9. A useful three-card result need not beat a single-GPU engine to justify publication.
0002-glm-quant-plan.md 0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)

0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)

Source: bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 (made with llama.cpp b11279, mainline-compatible; MTP layers kept at Q4_0). Unsloth GGUFs excluded: not loadable on mainline until re-uploaded.

LayoutQuantsSizeFit reasoning
1 card (24 GB VRAM + ~115 GiB usable RAM)IQ2_M, Q2_K120.6 / 125.7 GB (112 / 117 GiB)~21 GiB on GPU, rest in RAM
1 card, stretchIQ3_XXS138.8 GB (129 GiB)over RAM by ~10 GiB → mmap paging from NVMe; run once, expect slow, record it
3 cards (72 GB VRAM + RAM)IQ3_XXS, IQ4_XS138.8 / 177.7 GBIQ4_XS ≈ 165 GiB: ~66 GiB VRAM + ~99 GiB RAM; ~4-bit like every accepted GLM recipe

Order (disk is ~750 GB for everything, so sequential): IQ2_M → Q2_K → (IQ3_XXS) → delete the losing 2-bit → IQ4_XS. ik_llama.cpp + MTP runs on the same files.

Quality judged per SPEC §5 (tools gate, 0 parse failures over ≥100 calls, ≥80 % of reference solve rate in the uncapped eval).

Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Order: IQ2_M and Q2_K on 1 card → same files on 3 cards → IQ3_XXS on 1 card (tight resident candidate, not presumed paging) and 3 cards → IQ4_XS on 3 cards.
  • Fit is checked by measured placement at the stated context, per GPU; uneven splits watched.
  • Nothing is a finalist before the uncapped quality evaluation; deletion only of rejected candidates, after manifests.
  • MTP on/off is its own variable family; MTP use verified from engine logs and metrics.
  • Publish tool-call counts and reference solve counts next to every quality verdict.
0002-glm-quant-plan.review.md 0002-glm-quant-plan.review
  • AMEND. Keep the candidate quants, but correct the IQ3_XXS fit claim and make deletion depend on qualification. The strongest counter-argument is that weight capacity does not establish useful speed or quality: the spec already estimates one-card GLM at 8–13 tok/s, below the 15 tok/s registry floor.
  • IQ3_XXS is not automatically a paging experiment. Using the decision’s assumptions: 138.8 GB ≈ 129.3 GiB; subtracting 21 GiB GPU residency leaves 108.3 GiB in RAM, about 6.7 GiB below the stated 115 GiB usable budget. Treat it as a tight resident-memory candidate. Declare paging only after accounting for context, runtime allocations and actual tensor placement.
  • All fit claims need a specified context and measured placement. Approximate host-weight budgets are 91.3 GiB for IQ2_M, 96.1 GiB for Q2_K, and 99.5 GiB for three-card IQ4_XS. Their remaining host margins are roughly 23.7, 18.9 and 15.5 GiB respectively. These exclude additional runtime memory. Likewise, allocating 66 GiB across three cards leaves only 2 GiB per card on average; an uneven split can exhaust one GPU.
  • Budget bandwidth before promising 15 tok/s. Dual-channel DDR4-3200 has a theoretical maximum of 51.2 GB/s. At 15 tok/s, that permits at most 3.41 GB of DRAM traffic per generated token, with a smaller practical allowance. Measure effective bandwidth, expert placement and decode; VRAM capacity alone cannot show that the one-card route clears this constraint. Also, Q2_K is only 4.2% larger than IQ2_M—kernel efficiency and quality could easily determine the winner.
  • “~4-bit like accepted recipes” is precedent, not qualification. The spec’s reported sub-4-bit KLD of 0.28–0.38 makes quality a substantial risk. Establish the reference explicitly: exact EXL3 if loadable, otherwise the highest-bit GGUF that fits. If that is IQ4_XS, retain it through all candidate evaluations. A speed winner cannot be called a finalist before the uncapped quality evaluation.
  • Concrete alternative order: screen IQ2_M and Q2_K on one card, compare those files on three cards, then screen IQ3_XXS on both layouts and IQ4_XS on three cards. Run uncapped quality evaluation for promising configurations, then qualification for survivors. Give one-card IQ3_XXS an ordinary residency check before labelling it “stretch”; make three-card IQ3_XXS an explicit comparison point.
  • Replace “delete the losing 2-bit” with “delete a rejected candidate.” All four listed quants total 562.8 GB, leaving about 185 GB of the specified 748 GB free before Qwen weights, images and other artifacts. Build the combined inventory first. Preserve hashes and file references before deletion, and retain qualifying finalists under §9; being slower than another quant does not itself establish rejection.
  • Shared files do not establish shared MTP support. Verify that each pinned engine loads this exact revision and actually uses the retained MTP layer. Compare MTP off/on as a separate variable family; record accepted tokens/s, acceptance rate, settings and quality. Keep locally built ik_llama.cpp results experiment-only until its image provenance becomes eligible.
  • The decision’s quality summary omits essential acceptance conditions. Require the rolling 60-second decode floor, three qualification runs, cold-cache reset, no OOM/crash/repetition, and all six lab gates. Zero failures in 100 tool calls satisfies the spec but still gives an approximately 3% one-sided 95% upper bound on failure probability. The solve-rate test is also coarse: with four reference solves, all four are required; with five, four suffice. Publish these counts alongside occupied context and exact launch provenance.
0003-harness-interpretations.md 0003 — Harness interpretations of SPEC §5 (repetition, parse failures)

0003 — Harness interpretations of SPEC §5 (repetition, parse failures)

Context: the T06a harness was exercised for 10 min against the operator's daily Qwen3.8-27B (EXL3 3 bpw, SGLang): 73 tool calls, 0 malformed tool calls, but 12 repetition hits — genuine reasoning loops (the same multi-paragraph block 3× inside one reasoning field) on sympy-21612 at 60–83k occupied tokens. Under §5 as written, the operator's everyday model fails stability.

Interpretations implemented by the harness:

  1. Repetition, per field: the 64-token ×3 rule is evaluated within each generated field (reasoning, content, each tool call's arguments), not over the concatenation, because code drafted in reasoning and then emitted in a tool call trips the joined check without being a loop.
  2. Parse failures, strict: besides invalid tool-call JSON and unparsed tool-call markup, every agent format error counts — including a response truncated at max_tokens (8192) with no tool call. Each response counts once; causes are split in detail.
  3. Same-call streak (≥4 identical consecutive tool calls) resets at each task attempt.

Proposed (pending data, operator decides): keep §5 strict for this first pass and record all hits with causes; after T10/T20 produce real data, revisit whether "no repetition loops" should mean "no unrecovered loop" (a loop that ends the response by hitting max_tokens or ends the task) — not changed now.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • SPEC §5 amended: per-field repetition (cross-field diagnostic only), detector frozen for T10/T20, streak resets only on a fresh conversation (each task attempt starts a new mini-swe-agent conversation, so the harness already complies).
  • Three error categories: parse_failures (malformed tool calls only — the qualification criterion), output_truncations, agent_format_errors, all published.
  • No relaxation of "no repetition loops" now. Recovery and wasted-token figures to be published as supplementary data; any protocol revision later is prospective and does not re-grade earlier results.
  • Denominators always shown next to zero-failure claims.
0003-harness-interpretations.review.md 0003-harness-interpretations.review
  • AMEND. Keep the strict no-loop gate, but distinguish detector clarification from changes to eligibility. The strongest objection is that the harness simultaneously narrows repetition detection and broadens parse failures, while claiming to preserve §5.
  • Per-field repetition is defensible, but requires an explicit spec amendment. “Within one response” currently includes repetitions across fields. Adopt: “Within reasoning, content, or each tool-call argument field, independently, using the served model’s tokenizer.” Retain cross-field matches as diagnostics so the excluded cases remain auditable.
  • Do not label every agent-format error a tool-call parse failure. Invalid JSON and malformed tool markup qualify; an otherwise valid response exhausted at max_tokens without a tool call is a truncation or agent-protocol failure. Record tool_parse_failure, agent_format_error, and output_truncation separately. If all must disqualify, add that requirement explicitly to §5.
  • Resetting the same-call streak is appropriate only at an independent task attempt. Specify that a fresh agent conversation resets it; a retry continuing the same conversation does not. Otherwise attempt boundaries can conceal four consecutive identical calls.
  • The daily-model result does not justify relaxing the gate. Twelve genuine loops establish that this particular screening run fails the agreed criterion. They do not establish that the criterion is unsuitable for the two target models. The observed 60–83k occupancy also falls below the ≥128k requirement for “agentic-usable.”
  • Reject “unrecovered loops only” as the qualification alternative. Recovery does not erase wasted generation or latency. At exactly the specified decode floors, an 8,192-token response takes approximately 164 seconds for Flash-Next or 546 seconds for GLM, excluding prefill—27% or 91% of a ten-minute run. Publish recovery and wasted-token measurements as supplementary metrics while retaining zero detected loops for qualification.
  • The parse evidence is preliminary. Seventy-three calls falls short of the ≥100-call minimum. Under an independent binomial assumption, 0/73 failures still gives a one-sided 95% upper failure-rate bound of approximately 4.0%; 0/100 gives 3.0%. Report the denominator without presenting zero observed failures as strong reliability proof.
  • Concrete alternative: ratify the field boundary and separate error categories before target comparisons; freeze that detector version for T10/T20; preserve each offending response and count affected responses separately from detector matches. Use later data to propose a prospective protocol revision, with existing results retaining their original qualification status.
0004-core2-quarantine.md 0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)

0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)

Evidence: two hard resets on 2026-10-03 (04:57 near-idle, 06:56 during T10 model load). Each next boot logged an uncorrectable machine check in bank MC5 (Execution Unit) — CPU:18 MC5_STATUS[-|UE|MiscV|-|PCC|TCC|SyndV] and CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV]. CPUs 2 and 18 are SMT siblings of physical core 2. No OOM, no GPU Xid, no kernel panic text: the core faulted and the platform reset. Likely cause: per-core Curve Optimizer/PBO undervolt too aggressive for that core, or core degradation (RMA case if BIOS is stock; BIOS 5101).

Decision (operator, option 2): take CPUs 2 and 18 offline for the whole campaign (resets at reboot; hygiene.sh check refuses to benchmark while they are online). Every record carries host.cpus_online/cpus_offline. llama.cpp and ik_llama launches use 15 threads. Then validate with tools/stability/cpu_check.sh (all-core AVX2, per-core single-thread boost bursts, all-core openssl) and look for new machine checks.

Consequence for results: numbers are for a 15-core 5950X. CPU-bound routes (GLM experts on CPU) are affected ~6 %; GPU/PCIe-bound trellis decode should not be. Any result produced before this decision does not exist (T10 never completed).

Check result (2026-10-03 07:25–07:40)

cpu_check.sh 300 20 180 on the 30 online CPUs: 0 machine checks, no reset (log: logs/cpu_check-core2-offline.log). Not conclusive for the near-idle failure mode; the hygiene check keeps the quarantine enforced for every run.

Revision (2026-10-03 10:30) — quarantine lifted with a tripwire
  • Operator BIOS check: PBO was Auto (ASUS stock); Curve Optimizer set to all cores, positive 0 (= no offset). So the CPU was effectively at stock when it faulted; a weak core 2 remains the leading hypothesis.
  • TARGET_CPUS="2 18" cpu_check.sh 300 20 180 with all 32 CPUs online: all-core AVX2, per-core bursts, 5 min sustained on CPU 2 and 18 each, 5 min on/off bursts on each, all-core openssl — 0 machine checks, no reset.
  • Not conclusive (both crashes came after hours of uptime). Core 2 is back online for full capacity; hygiene.sh check refuses to benchmark and re-offlines CPUs 2,18 if any machine check appears in the current boot; every record carries host.machine_checks_this_boot. llama.cpp/ik_llama launches back to 16 threads.
  • T10 ran on 15 cores (recorded in its host block). Trellis decode is PCIe-bound; one 16-core re-check is planned.
  • If it recurs at stock: keep core 2 offline for the campaign; RMA candidate.
Revision (2026-10-04 08:05) — permanent quarantine
  • Third fatal reset with core 2 online at stock: 2026-10-04 06:14, ~20.5 h into the boot, during the 16-core T10 re-check model load (CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], logged at the 06:54 boot). Same pattern as the 2026-10-03 06:56 crash (Flash-Next load) after a long uptime.
  • Core 2 stays offline for the rest of the campaign, enforced at boot by local-ai-bench-quarantine.service (tools/box/install.sh) and by hygiene.sh check (CPU_QUARANTINE defaults to "2 18"). All llama.cpp/ik_llama launches use 15 threads. The 16-core T10 re-check is dropped; all results are for a 15-core 5950X.
  • Stock settings + repeated MC5 UE on one core: AMD RMA candidate (operator's call).
0005-ik-llama-glm-tool-parsing.md 0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)

0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)

Finding (T22b, IQ2_M, 1 card): the model emits well-formed GLM tool calls (<tool_call>bash<arg_key>command</arg_key><arg_value>…</arg_value></tool_call>) but ik_llama's server returns them as plain content with empty tool_calls: lab tools gate fails; stability run 0 tool calls / 47 parse failures (unparsed_markup). ik_llama builds tool parsers from the chat template (common/chat-auto-parser*); mainline llama.cpp b11312 parses the same GGUF correctly (tools gate passed in T20b). No matching ik_llama issue/PR found (search 2026-10-04).

Speed is unaffected and recorded: decode 10.0 tok/s (mainline 9.6), prefill 151 lab / 203 TTFT-8k (mainline ~80).

Decision: keep the ik_llama route as speed evidence only (experiment-only, fails §5 tools); do not patch ik_llama in this campaign — single-card GLM is ~10 tok/s on every route, below the 15 tok/s gate, so tool parsing would not change the outcome. Phase 3 option: upstream bug report with the captured response (artifacts/runs/…ik_llama…stability/proxy/parse_failures/).

3-card follow-up (2026-10-04, GPU2-first order, decision 0006)
  • No MTP, --fit-margin 4096: lab decode 13.4 / 14.0, agentic 13.3 tok/s steady (best GLM decode on this box), prefill 185–283; tool calls still unparsed (68 agent-format errors), tools gate fails.
  • MTP, --fit-margin 7168 (4096 OOMed creating the MTP context): lab decode 10.2 (acceptance 51 %), agentic 15.6 whole-run before the server crashed mid-run. The extra margin for the draft context pushes experts to host RAM.
  • Route parked: experiment-only speed evidence. The most interesting number for 0xSero is 13–14 tok/s without MTP; a tool-parser fix in ik_llama plus an attested image would be needed before it could be a 3-card candidate.
0006-pcie-layout.md 0006 — PCIe lane layout: what can be reconfigured, and what it buys

0006 — PCIe lane layout: what can be reconfigured, and what it buys

Operator question (2026-10-04): can the PCIe lanes be reconfigured to get better model performance?

Measured topology (lspci, nvidia-smi, T05a)
DeviceAttachLinkMeasured H2D
GPU0 3090 04:00.0X570 chipset, behind a Gen4 x4 uplink shared with the KC3000 1 TB NVMe (root + all model files), NIC, BMC, SATA, USBGen4 x8 to the chipset6.0 GB/s
GPU1 3090 0b:00.0CPU lanes, slot 2 (x8/x8 split) — drives the displayGen4 x8not measured (display)
GPU2 3090 0c:00.0CPU lanes, slot 1 (x8/x8 split)Gen4 x813.5 GB/s
990 EVO Plus 4 TBCPU lanes (M.2_1, Gen4 x4)unused: holds another OS install

Already optimal and not a lever: every link trains at Gen4 (16 GT/s); Resizable BAR is on (32 GB BAR1); ASPM is disabled on every GPU link. AM4 has 24 usable CPU lanes (16 slots + 4 M.2 + 4 chipset uplink); bifurcation or risers cannot add lanes, so the only real choices are which card sits where and how many cards share the 16 slot lanes.

Where PCIe bandwidth limits each route (evidence from our runs)
  1. Flash-Next trellis (experts streamed from host RAM to one GPU): decode is PCIe-bound. 0xSero's data: ~26 GB/s → 62.7 tok/s, 14–16 GB/s → 53.2. Ours at 13.5 GB/s: lab 49–55, kit C1 46.9, agentic 43. x16 should add ~15–25 %.
  2. llama.cpp/ik_llama prefill with experts in RAM is PCIe-bound (CPU-resident weights are copied to a GPU for large batches). Evidence: on GPU2, raising -b/-ub 512 → 2048 lifted GLM IQ2_M prefill 85 → 200 tok/s (each copy serves 4× more tokens). And 3 cards prefilled slower than 1 card (158 vs 200), consistent with the copy going to CUDA0 = GPU0, the 6 GB/s chipset card.
  3. llama.cpp decode with experts on the CPU is DDR4-bound, not PCIe-bound (GLM ~9.5 tok/s on 1 card, 12.3 on 3 cards, from less RAM-resident weight). PCIe changes nothing here.
Options
#ChangeCostExpected effect
S13-card llama.cpp with the CPU-attached GPU2 as CUDA0 (CUDA_VISIBLE_DEVICES=2,1,0 in the container)none3-card prefill back to ≥ 1-card level (~200+), no decode change
S2Larger ubatch (-b/-ub 4096) on llama.cpp routesnone (more VRAM for compute buffers)prefill up to ~1.5–2× again; watch VRAM (the IQ2_M 3-card run already OOMed on CUDA0)
P1One card alone in slot 1 at x16 (T12, already planned)operator moves cardsFlash-Next decode +15–25 % (the only route to ≥ 50 headline at full context); llama.cpp prefill ~2× vs x8; GLM exact mode at the bandwidth its 12.6 tok/s reference used
P2Plug the monitor into GPU0 (chipset card) instead of GPU1move one cableGPU1 + GPU2 (both CPU x8) free for 2-card runs without going headless; desktop stays usable during single/2-card work
P3Model files on the CPU-attached 990 EVO Plus instead of the chipset KC3000needs the operator's OK (other OS lives there)removes model loads and Flash-Next n-gram NVMe reads from the chipset uplink that GPU0 also uses; small effect except for 3-card runs with GPU0
P42 cards on CPU lanes (x8/x8), third card removed or idlenone (GPU1+GPU2)avoids the 6 GB/s card; less VRAM (48 GB) so more experts in RAM
Decision (proposed)
  • Run S1 and S2 now in the current layout (software only, attributable, one variable family each), on GLM IQ2_M: 3-card GPU2-first at ub2048 (vs the existing GPU0-first record), then 1-card GPU2 at ub4096 (vs ub2048).
  • Keep P1 (T12) as the main hardware lever; in the x16 session also rerun the best GLM llama.cpp 1-card config so the x8→x16 effect on prefill is measured directly.
  • Recommend P2 to the operator at the T12 visit (same visit, one cable). P3 only if the operator wants to free the 990 EVO Plus; not worth touching another OS for this campaign.
  • Publish this as a site methodology section with the before/after records.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted, plus operator rule
  • Bandwidth effects are hypotheses until measured. Expected-gain numbers above are estimates, not claims; x16 is the principal hardware hypothesis for Flash-Next, not a guarantee of qualification.
  • S1 is a fresh paired comparison, both headless, identical except card order: IQ2_M 3-card ub2048 with a 2 GiB --fit-target margin (the default-margin run OOMed on CUDA0 after 62 min), GPU0-first vs GPU2-first (CUDA_VISIBLE_DEVICES=2,1,0; the container sees the cards in PCI order). Every TTFT step now logs per-GPU PCIe rx/tx (nvidia-smi dmon -s t) to show which card receives the weight copies. Decode changes are reported, not assumed.
  • S2: 1-card GPU2 ub2048 vs ub4096, both at 15 threads (the old ub2048 record ran 16 threads), peak VRAM and OOMs recorded.
  • P1: rerun the best 1-card GLM llama.cpp config at x16 with the same bandwidth procedure (x8 control exists).
  • Operator rule (2026-10-04): GPU0 is used last. Single-card runs use GPU2; 2-card runs GPU2+GPU1; 3-card runs order the cards GPU2, GPU1, GPU0. Pending S1 evidence, all new multi-card launches use the GPU2-first order.
  • P2 offered to the operator at the T12 visit; P3 deferred (would need a §9 exception and touches another OS).
S1 result (2026-10-04 12:15–13:05 AEST) — card order, GLM IQ2_M 3-card ub2048, 2 GiB fit margin, headless
GPU0 first (control)GPU2 first (CUDA_VISIBLE_DEVICES=2,1,0)change
TTFT prefill 8k / 32k159 / 169 tok/s240 / 242 tok/s+51 % / +43 %
lab prefill153 tok/s218 tok/s+43 %
lab decode C112.2 tok/s11.8 tok/s−3 %
PCIe rx during TTFT (avg / peak)GPU0 3.9 / 6.5 GB/s (link-saturated), others ~0.3GPU2 5.7 / 14.3 GB/s, others ~0.5

The host→device weight copies for prefill go to CUDA0; putting a CPU-lane card there lifts prefill ~1.5×. Decode is within noise of unchanged (slightly lower). The GPU2-first config also finished a clean 10-min agentic run (11.3 tok/s, 24 tool calls, 0 parse failures, 0 loops) and passed the post-run gates without the OOM that hit the default margin. Rule confirmed: GPU0 last.

S2 result (2026-10-04 13:05–14:23 AEST) — ubatch, GLM IQ2_M 1 card GPU2, 15 threads
ub2048 (control)ub4096change
lab prefill201 tok/s288 tok/s+43 %
TTFT prefill 8k / 32k213 / 222321 / 316+51 % / +42 %
lab decode C19.59.6unchanged
VRAM at ready21.9 GB22.9 GB+1 GB, no OOM in a 10-min agentic run

The 15-thread control matches the earlier 16-thread ub2048 record (9.5 / 200), so the core-2 quarantine costs nothing measurable on this route. Gain > 3 %, so the search continues: ub8192 on 1 card, and ub4096 on the GPU2-first 3-card config.

S2 step 2 (2026-10-04 16:13–16:30 AEST) — ub8192, 1 card GPU2

lab prefill 370 (+28 % vs ub4096), TTFT 384 / 404, but lab decode 9.1 (−5 %): the larger compute buffer makes --fit keep more experts in host RAM (VRAM at ready 22.2 GB). Prefill and decode now trade off, so the batching search stops here: ub4096 is the balanced setting (decode unchanged, prefill +43–51 %); ub8192 is the prefill-maximising variant, published as such.

  • 3-card GPU2-first, ub2048 → ub4096 (16:30–16:58): lab prefill 218 → 258 (+18 %), TTFT 240 → 298 (+24 %), lab decode 11.8 → 11.2 (−5 %). Same prefill/decode trade-off as 1 card at ub8192; both published, no further steps.
0006-pcie-layout.review.md 0006-pcie-layout.review
  • AMEND. Proceed with S1, a bounded S2 test, and T12, but describe the bottlenecks and speedups as hypotheses. The strongest counter-argument is that the current runs change batching, weight placement, and GPU count; they do not isolate PCIe bandwidth.
  • Flash-Next’s +15–25% estimate is plausible; “PCIe-bound” overstates the evidence. The reference gain is only 17.9% despite bandwidth rising from 14–16 to 26 GB/s. A simple fixed-work Amdahl model attributes roughly 33–39% of baseline time to transfers, predicting 20–25% from doubling bandwidth. That model remains unverified across machines and contexts.
  • x16 does not establish qualification. Applying +15–25% to the measured agentic 43 tok/s gives 49.5–53.8 tok/s, and averages cannot prove every required rolling window clears 50. Replace “the only route to ≥50 headline” with “the principal hardware hypothesis”; qualification still requires the same config to pass occupied-context, prefill, quality, and stability requirements.
  • The prefill explanation is not established. Increasing batch size fourfold produced 2.35× throughput, from 85 to 200 tok/s. Transfer amortization, better GPU utilization, and execution overhead can all explain this. Likewise, 158 versus 200 tok/s across three versus one GPU does not identify CUDA0 as the cause. Replace “PCIe-bound” and “~2× at x16” with measured observations and an explicit hypothesis.
  • Make S1 a fresh paired comparison. Run GPU0-first and GPU2-first at ub2048 with identical weights, offload settings, tensor split, context, and cache state; both must be headless. Select devices by UUID, record their logical CUDA mapping, and verify which GPU actually receives the expert transfers. Reordering devices alone does not demonstrate the claimed routing change. Report decode changes rather than assuming none.
  • Bound S2 by memory and measurement. Compare one-card GPU2 ub2048 with ub4096 while holding the other settings fixed; record peak VRAM and OOMs. Remove the promised “1.5–2× again.” If both batch and ubatch change, identify the comparison as a batching-family change without attributing its effect specifically to ubatch.
  • Keep P1, with a matching x8 control. Run the exact same best one-card GLM configuration before and after the move, alongside bandwidth measurements under a defined, identical procedure. The 12.6 tok/s exact-mode reference already used x16 and remains 16% below the 15 tok/s floor; matching its link does not imply registry eligibility.
  • P2 is useful, subject to the display rule. Moving the display to GPU0 enables display-free GPU1/GPU2 comparisons. Three-card tests still require logout and display: off; the x16 benchmark must also use a GPU that is not driving the display.
  • Defer P3 and public publication as proposed, but clarify scope. Storage contention has not been measured, and changing active-model storage would require an explicit exception to §9. Put the methodology and every before/after result in the private preview now; public publication remains Phase 3 after operator and log review.
0007-glm-current-layout-finalists.md 0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs

0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs

Facts (all current-layout GLM records, 2026-10-03/04)
RouteCardsLab decodeLab prefill10-min run (tool calls / parse failures / loops / solved)
llama.cpp IQ2_M ub409619.628827 / 0 / 1 / 1 of 2
llama.cpp IQ2_M ub2048, GPU2 first311.821824 / 0 / 0 / 1 of 2
llama.cpp IQ2_M ub2048, GPU0 first312.315826 / 0 / 0 / 1 of 2 (server later OOMed)
llama.cpp IQ3_XXS ub2048310.413222 / 0 / 1 / 1 of 2
llama.cpp IQ4_XS ub2048, GPU2 first38.315625 / 0 / 0 / 1 of 2
llama.cpp Q2_K / IQ3_XXS19.2 / 8.378 / 650–1 solved
ik_llama IQ2_M (± MTP)18.2–10.0 (MTP agentic ~12.6)~1500 tool calls: GLM tool calls not parsed (0005)

Every route is below the 15 tok/s registry gate on 1 card and on 3 cards (best 12.3). Decode is DDR4-bound: more cards only help by holding more experts in VRAM. Pending in the queue: IQ2_M 1-card ub8192 and 3-card ub4096 (prefill-only), ik_llama 3-card ± MTP (speed evidence only).

Proposal
  1. No current-layout GLM finalist. §5 qualification requires decode ≥ 15 tok/s in every window, so the 3 × 10-min qualification runs would fail by construction. T25a records "no finalist: below the floor" for every current-layout config instead of running them. All configs stay published as documented experiments (§6 Phase 2).
  2. One uncapped quality evaluation, of the best mainline weights, in the x16 session. Solve rate depends on weights, engine and sampling, not on the PCIe layout; the layout only changes how long the evaluation takes. So the T07 candidate evaluation of llama.cpp IQ2_M runs on the single x16 card next to the T23 reference (exact mode, or the highest-bit GGUF that fits if exact mode cannot load), with the identical task list, sampling and 60k-token budget. Each evaluation is estimated at 4–7 h (measured ~3–4k completion tokens per 10 min).
  3. No uncapped evaluation for IQ3_XXS / IQ4_XS / Q2_K / ik_llama. They are slower than IQ2_M and cannot become candidates; their 10-min screening solve counts are published labelled "screening only". IQ4_XS only fits on 3 cards, so measuring it would cost ~6 h of current-layout time for an informational number.
  4. Consequence: T12 (card move) can happen as soon as the current queue finishes (~3.5 h), not after ~1 day of current-layout quality runs. After the queue, IQ3_XXS and IQ4_XS are marked rejected and deleted (manifests kept) to make room for the x16 weights (Flash-Next 2.05/4.05 bpw: 63/108 GB; GLM exact EXL3: 125 GB).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Below-floor is measured, not inferred from lab C1: every current-layout llama.cpp config with a 10-min run has a measured rolling 60-s generation-only minimum below 15 (IQ2_M 3-card 11.0–11.4, IQ2_M 1-card 8.8–9.1, IQ3_XXS 3-card 9.9, IQ4_XS 3-card 7.8). Configs without a stability run (prefill-only S1/S2 controls, ub8192) are recorded as "qualification skipped: subfloor lab C1; rolling floor not measured". T25a closes with no current-layout finalist once the pending queue (ik_llama 3-card ± MTP, ub follow-ups) is in; ik_llama stays experiment-only (0005).
  • Quality evaluation is conditional. At x16: measure IQ2_M 1-card first; only a config that qualifies (§5 incl. the rolling floor, ≥ 100 cumulative tool calls, post-run gates) gets the uncapped T07 candidate evaluation. An optional quality study of a non-qualifying config is labelled informational. Quality results apply to the tested configuration; layout independence is not assumed.
  • Runtime estimate corrected: the 60k budget is per task; worst case 2.5–3.3 h per task, 12–33 h per evaluation (less when tasks submit early). Budget accordingly; this is another reason to evaluate qualifiers only.
  • Labels: IQ4_XS, IQ3_XXS (3-card), Q2_K: "rejected for speed, quality unmeasured". IQ2_M 3-card GPU0-first: OOM after 62 min (default fit margin). IQ2_M/IQ3_XXS repetition hits are published with the run.
  • Storage: keep IQ2_M (x16 rerun) and IQ3_XXS (highest-bit GGUF that fits 1 card = fallback reference if exact mode cannot load); delete IQ4_XS after the queue, manifests and hashes kept for reacquisition.
0007-glm-current-layout-finalists.review.md 0007-glm-current-layout-finalists.review
  • AMEND. Finish the queue and move to x16 without spending a day on current-layout quality runs. The strongest objection is that the proposal treats lab C1 as proof of failing a different metric, then schedules a “candidate” quality evaluation for a configuration that has not qualified.
  • “Fail by construction” is unsupported. Lab decode measures the first 30 seconds; §5 requires generation-only decode over rolling 60-second windows, ignoring the first minute. Best C1, 12.3 tok/s, is 18% below the floor and needs 22% uplift to reach 15. That warrants deprioritization, but not a claimed measurement of the floor. Use existing screening traces to establish a failing window; otherwise record “qualification skipped: subfloor lab C1; rolling-floor eligibility unverified.”
  • Make x16 quality evaluation conditional. §5 says “Finalists only.” First measure IQ2_M on x16; if promising, complete qualification, including ≥100 cumulative tool calls and all six post-run gates. Run the uncapped comparison only if it qualifies. If it remains below 15, finish it as a documented experiment under Phase 2. Any optional quality study should be explicitly labelled informational.
  • Layout independence is a hypothesis, not a guarantee. Fixed weights and token budgets remove much of the speed bias, but device placement can change numerical execution and sampled trajectories; timeouts, OOMs and truncation can also affect outcomes. The x16 evaluation directly supports the tested x16 configuration. Transfer its quality result to another layout only with matching engine, settings and evidence of equivalent behavior.
  • IQ2_M is the speed leader, not established “best weights.” The 10-minute results cannot rank quantization quality. IQ4_XS could be slower yet substantially more accurate—a relevant possibility given §12’s sub-4-bit quality risk. Skipping its uncapped evaluation is reasonable because it has no demonstrated eligibility; label it rejected for speed, with quality unmeasured.
  • The runtime estimate needs its budget scope stated. At 3–4k completion tokens per 10 minutes, exhausting 60k tokens per task takes 2.5–3.3 hours per task. Across 5–10 tasks, that is 12.5–33 hours per evaluation, or 25–67 hours for candidate plus reference, assuming similar throughput. Early completion can reduce this. If 60k means an aggregate budget, specify the per-task allocation; “4–7 hours” currently lacks a clear basis.
  • Keep T25a provisional until the queue finishes. Pending ik_llama 3-card/MTP measurements could change the speed ranking, although parsing and image eligibility remain blockers. Existing IQ2_M repetition and GPU0-first OOM evidence also need explicit failure labels. Faster decode alone does not make these configurations finalists.
  • Treat “DDR4-bound” as a working diagnosis. The table does not isolate memory bandwidth from PCIe, placement, batching or engine overhead. The one-card 9.6 tok/s result needs 56% uplift to reach 15, so an x16 rescue is uncertain. Measure it before committing to expensive quality work.
  • Concrete replacement: finish pending measurements → close current-layout records with measured failures or justified skips → move cards → measure and qualify x16 routes → evaluate qualifying candidates against the specified reference. Delete rejected weights after preserving hashes and immutable references, while reserving or budgeting reacquisition of the highest-bit GGUF fallback. Keep results private until Phase 3.
0008-flashnext-3card-llamacpp.md 0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate

0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate

Evidence (T11, 2026-10-04 15:38 AEST, IQ4_XS, GPU2,GPU1,GPU0 order, 2 GiB fit margin, -b 2048 / -ub 512 defaults)
  • Lab: all six gates pass, decode C1 57.1 / 56.9 tok/s (before / after the agentic run), lab prefill 608 / 605.
  • TTFT prefill: 987 tok/s at 8k, 852 at 32k.
  • 10-min agentic run: 70 tool calls, 0 parse failures, 0 loops, 1 of 2 solved, whole-run decode 45.4, lowest 60-s window 38.6 (fails the 50 floor; decode falls as the context grows).
  • VRAM at ready 65.5 GB of 72; weights 97.7 GB, so ~30+ GB (incl. the n-gram embedding table) stays in host RAM.
  • For comparison, the 1-card trellis/SGLang baseline at x8: lab 49–55, kit C1 46.9, agentic 43.

Per SPEC §6 a 3-card config is a separate, clearly labelled recipe; llama.cpp routes are judged by the registry lab and §5 (the kit's SGLang-only panel cannot score a GGUF, so it is "kit band not measurable").

Proposal (current layout, before T12; one variable family per step, stop rule 3 × < 3 %)
  1. Re-download IQ4_XS (deleted at 16:13 per plan; manifest kept, 97.7 GB, ~15 min).
  2. Prefill: -b/-ub 2048, then 4096 (GLM showed +135 % then +43 %; target lab prefill > 1000).
  3. Decode floor: the floor fails late in the run as context grows. One quant step down to IQ3_M (93.2 GB) or IQ3_XXS (88.0 GB) puts more experts in VRAM; run the better prefill setting from step 2 on it. Quality cost is unmeasured by the kit for GGUF; report solve counts and parse failures, label "quality: screening only".
  4. If a config reaches lab decode ≥ 50, lab prefill > 1000 and holds the 50 floor in a 10-min run, it becomes a 3-card finalist: §5 qualification (3 runs incl. cold and max context, ≥ 100 tool calls) before T12.
  5. Disk: the x16 downloads (296 GB) wait until this route is done and its rejected GGUFs are deleted.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Route is experiment-only unless an accepted scoring path establishes the Flash-Next kit band for a GGUF; kit band recorded as unavailable, registry/headline eligibility unestablished. The T11 config is a failed screening config (floor −29.5 %, lab prefill −65 % vs targets), reported per constraint.
  • Batching steps, predeclared, judged by official lab prefill (decode and VRAM regressions checked): A -b 2048 -ub 2048, B -b 4096 -ub 4096, both vs the existing -b 2048 -ub 512 record. (GLM evidence says the ubatch sets the host→GPU copy amortisation, so b and ub move together.)
  • Before any quant change: decode vs occupied context from the existing T11 agentic trace (per-request generation speed against prompt tokens). IQ3_M is screened only if that shows a residency/transfer limit, not attention cost; its step is judged by the minimum rolling-window decode at matched occupancy. Expert/n-gram placement and memory use recorded from the server log.
  • Qualification only for a speed-passing config, naming its maximum occupied context (≥ 4k headroom), cold resets, post-run gates, ≥ 100 tool calls cumulative. Phase timebox kept; x16 is not deferred beyond this route's steps.
Decode vs occupied context (T11 3-card agentic trace, 55 requests, generation-only tok/s per request)
prompt tokensper-request decode
1.5k–6k (django task)35–51, typical ~42
6k–20k30–49, typical ~40
20k–31k30–41, typical ~38
31k–39k28–38, typical ~33

Two separate gaps to the 50 floor: a base gap (~42 tok/s already at 2–6k tokens, vs lab C1 57 measured over one long generation; agentic requests are short, 70–600 tokens) and a context slope of roughly −20 % from 5k to 39k. One quant step (IQ3_M, −4.6 % of bytes; IQ3_XXS −9.9 %) cannot close a 25–30 % gap even if every saved byte moved experts to VRAM. IQ3_M step dropped; the predeclared batching steps still run (prefill evidence, cheap). After them the route closes as experiment-only and its GGUF is deleted for the x16 weights.

Batching result (2026-10-04 18:00–18:23 AEST) — route closed
-b / -ublab decode C1lab prefillTTFT 8k / 32k
2048 / 512 (T11 record)57.1608987 / 852
2048 / 204844.3 (−22 %)717 (+18 %)1067 / 931
4096 / 409635.0 (−39 %)713980 / 881

Larger ubatches buy little prefill and cost a lot of decode: the bigger compute buffers make --fit move experts to host RAM. No setting reaches lab prefill > 1000 while keeping decode ≥ 50, and the agentic floor already failed at the default. Route closed: experiment-only, best config = the default-batch T11 record (all six gates, lab 57.1, agentic floor 38.6). GGUF deleted for the x16 weights (manifest kept).

0008-flashnext-3card-llamacpp.review.md 0008-flashnext-3card-llamacpp.review
  • AMEND — retain this as a separate 3-card experiment, but remove the promised path to registry eligibility until quality is measurable. The strongest counter-argument is contractual: §5 explicitly requires the Flash-Next kit band for both the headline and registry PR. “Kit band not measurable” cannot establish a pass. §12’s lab-only tooling note creates an ambiguity; it does not explicitly waive that requirement.
  • T11 is a failed screening config despite passing all six lab gates. Its worst window needs 29.5% improvement to reach 50 tok/s; lab prefill needs approximately 65% improvement to exceed 1000. Lab C1 at 56.9–57.1 does not compensate for either failure. Report these as separate constraints.
  • The prefill experiment is reasonable; the GLM gains are insufficient evidence for its expected payoff. Specify exact settings rather than “-b/-ub 2048, then 4096.” A concrete sequence is baseline -b 2048 -ub 512, then -b 4096 -ub 512, then -b 4096 -ub 1024, measuring identical occupied contexts and fresh prompts. Select using official lab prefill, and check for decode or memory regressions.
  • The quantization hypothesis needs a residency measurement. IQ3_M saves 4.5 GB (4.6%); IQ3_XXS saves 9.7 GB (9.9%). Against roughly 30 GB remaining on the host, those represent at most approximately 15% / 32% reductions, assuming every saved byte reduces host residency. They do not prove that more active experts become GPU-resident or that transfers shrink proportionally. Record expert placement, n-gram placement, and actual memory use.
  • “Decode falls as context grows” establishes correlation, not the bottleneck. Before changing quant, report generation speed against occupied context, cache state, and GPU placement. Repeat controlled requests at representative early and late occupancies using IQ4_XS. Otherwise a smaller quant could obscure a context-dependent attention or transfer limitation.
  • Solve counts cannot replace the required quality comparison. One of two solved tasks is weak screening evidence, especially under a wall-clock cap where faster configurations receive more generation opportunities. Keep “quality: screening only,” but explicitly mark the kit band unavailable and registry/headline eligibility unestablished for every GGUF lacking valid panel scores.
  • Qualification must name the context it qualifies. Step 4 needs a claimed maximum occupied context, ≥4k generation headroom, the defined cold-cache resets, and post-run lab gates for each qualification run. “Agentic-usable” additionally requires ≥128k occupied tokens. Count ≥100 tool calls across screening, qualification, and any extra runs, as specified; publish truncations and agent-format errors too.
  • Make the search bounded and the stop rule meaningful. Predeclare which metric determines each step’s “gain”: official lab prefill for batching, minimum rolling-window decode at matched occupancy for quantization. A gain in one metric cannot excuse losing another acceptance criterion. Keep the phase’s 2–3-day timebox; disk sequencing alone should not defer x16 indefinitely.
  • Concrete revised decision: restore pinned IQ4_XS; run the two explicit batching changes; establish the context-dependent slowdown; screen IQ3_M, then IQ3_XXS only if justified. Run full stability qualification only on a speed-passing configuration. Unless the existing accepted scoring path establishes the kit band, publish the route as experiment-only, preserve its records, delete rejected weights under §9, and proceed to x16. Every 3-card record must include GPU UUIDs and display: off.
0009-no-x16-session.md 0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)

0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)

Operator, 2026-10-04 19:15 AEST: skip the x16 test; the expected difference is judged marginal. T12 (card move) and T05b are cancelled. The operator's call stands; the estimate it overrides is recorded for the reader: 0xSero's reference gives +17.9 % decode for 14–16 → ~26 GB/s, and our x8 baseline (lab C1 49–55, kit C1 46.9, agentic 43, floor not held) sits just under the 50 tok/s target, so reaching it now depends on tuning at x8.

Re-plan (single card = GPU2, CPU-lane x8, 13.5 GB/s; GPU1 measured identical)
  1. T13 PR #144 rerun at x8 against image f77c72f6 (lab, kit sweep with early exit off, TTFT, panel, 10-min run). Proof for 0xSero is "x8 on this box"; whether he accepts it is asked in Phase 3.
  2. T15 brought forward: the main recipe on EXL3 2.05 and 4.05 bpw, only the weights changed. At x8 the expert bytes per token are the bottleneck, so 2.05 bpw is the most likely route to ≥ 50 sustained; the kit panel decides whether it stays in the quality band (required for the headline and registry PR). 4.05 bpw is the quality anchor.
  3. T23 GLM exact mode on GPU2 at x8 (route test + quality reference; may not load: ~119 GB pinned on ~125 GiB).
  4. T14a/T14b grid on the best quant from 1–2 (expert cache GB × context × KV format; then prefill chunk and n-gram tier), designed as its own decision once 1–3 are in. T16 qualification and headline follow on the winner.
  5. GLM IQ2_M x16 rerun dropped; IQ3_XXS kept as the fallback reference.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Recorded as a deliberate restriction of the search, not evidence that x16 is marginal. Gap to target at x8: kit C1 46.9 → 50 needs +6.6 %; agentic 43 → 50 needs +16.3 %; there is currently no qualifying Flash-Next result.
  • Order: T13 at 3.05 → matched 2.05 / 4.05 screens (same launch, same context) → kit panel as a filter (top-1 ≥ 0.987, KL ≤ 0.0013) before any headline time → bounded tuning of in-band candidates, 3.05 included. Out-of-band results are published with kit eligibility, never as the registry path. "Bytes per token" is a hypothesis; transfer/cache evidence is recorded where the engine exposes it.
  • Acceptance unchanged: every eligible rolling 60-s window ≥ 50 and lab prefill > 1000 on the same config at the claimed occupied context. If no x8 config meets speed and quality together, that outcome is published as is.
  • T23 at x8: exact mode may be the GLM quality reference even below 15 tok/s (not a recipe); peak host memory and any load failure recorded (~110.8 GiB pinned vs ~125 GiB MemTotal). If it cannot load, the reference is the highest-bit GGUF that actually fits one card here, determined by a load test, not assumed to be IQ3_XXS.
  • Timebox and the 3 × < 3 % stop rule stay. Cancelled tickets (T12, T05b, x16 parts of T13/T14/T23) are recorded as skipped with the operator's reason.
0009-no-x16-session.review.md 0009-no-x16-session.review
  • AMEND. Respect the operator’s cancellation of x16, but describe it as a deliberate restriction of the search—not an evidence-backed conclusion that the gain is marginal. The strongest counter-argument is that the cited 17.9% uplift could change eligibility, whereas lower quantization risks losing the required quality band.
  • The target gap is material and measurable. Kit C1 46.9 → 50 requires +6.6%; agentic average 43 → 50 requires +16.3%, and reaching that average would still not prove the rolling floor. Applying the reference uplift illustratively gives 55.3 and 50.7 tok/s, respectively. These are scenarios, not predictions for this machine or workload.
  • Retain 3.05 bpw as a tuning candidate. The proposed sequence risks selecting an out-of-band 2.05 model or a slower 4.05 model before exploring the original quant. Concrete alternative: T13 at 3.05 → matched 2.05/4.05 screens → quality filter → bounded tuning of qualifying candidates, including 3.05.
  • Treat “expert bytes per token are the bottleneck” as a hypothesis. Neither the bandwidth measurement nor the quoted speeds establishes how much decode time is exposed transfer, compute, synchronization, or cache misses. Moving 3.05 → 2.05 bpw reduces nominal weight bits by 32.8%; under a purely weight-transfer-limited model that suggests about 49% more throughput. Metadata, cache residency and other costs can substantially change that result. Record transfer/cache evidence where available before assigning causality.
  • Use the panel early as a search filter. Require top-1 ≥ 0.987 and KL ≤ 0.0013 before spending headline qualification time on a quant. Publish outside-band results and retain their kit eligibility, but do not make them the sole path toward the registry target. Keep occupied context and other settings matched for the initial weight comparisons.
  • Preserve the actual acceptance test. Lab C1 over 30 seconds, a kit score, and a whole-run agentic average cannot establish a sustained floor. The winner needs every eligible rolling 60-second window ≥ 50, all qualification requirements, and lab prefill > 1000 on that same config, at the claimed occupied context. The existing “floor not held” evidence means there is currently no qualifying Flash-Next result.
  • T23 is useful at x8, but distinguish reference eligibility from recipe eligibility. Exact mode may serve as the GLM quality reference even below 15 tok/s; it then remains ineligible for a registry recipe. 119 decimal GB ≈ 110.8 GiB, leaving roughly 14.2 GiB against the stated 125 GiB MemTotal before runtime and system costs. That is plausible but unproven headroom; record peak host memory and any load failure.
  • IQ3_XXS cannot automatically become the GLM quality reference. The spec requires the highest-bit GGUF that fits when exact mode cannot load. Keep IQ3_XXS as a performance fallback, but separately determine the fitting quality reference and run the identical-task, identical-budget, uncapped comparison required by §5.
  • Make the amendment auditable and bounded. Update §6, §11 and the x8 risk discussion; record cancelled tests as skipped with the operator’s reason. Keep the phase timebox and diminishing-return stop rule. If no x8 config meets speed and quality together, publish that outcome explicitly rather than relaxing acceptance. The revised deliverable is the best qualifying result found within the x8 search.
0010-cpu-boost-off.md 0010 — CPU boost off for the rest of the campaign

0010 — CPU boost off for the rest of the campaign

Evidence (2026-10-04 19:48 AEST): hard reset during the T13 Flash-Next SGLang model load with core 2 already offline. The next boot decoded CPU:1 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], "uncorrected error caused a data fabric sync flood" — the same bank, status and syndrome as the core-2 faults (0004), now on core 1. Tally of fatal resets: 5, of which 3 during a Flash-Next SGLang load (expert repacking: heavy multi-threaded host work) and 2 after long uptime. Temperatures at idle are normal (Tctl ~56–60 °C). A second core failing the same way points to the CPU (or its voltage under boost transients) rather than one bad core.

Decision: disable Precision Boost from Linux (/sys/devices/system/cpu/cpufreq/boost = 0, all cores capped at the 3.4 GHz base clock, lower voltage), applied at boot by local-ai-bench-quarantine.service and immediately on 2026-10-04 20:12. Core 2 stays offline. Every record now carries host.cpu_boost and host.cpu_max_mhz.

Consequence: CPU-bound work (GLM experts on CPU, model loads) gets slower; PCIe-bound trellis decode should not. Records before 20:12 ran with boost on and are labelled by that field's absence (= boost on). If faults continue with boost off, the CPU is not usable for unattended runs and the remaining work pauses for the operator (RMA).

0011-flashnext-x8-tuning-grid.md 0011 — Flash-Next tuning at x8 (T14): grow the expert cache

0011 — Flash-Next tuning at x8 (T14): grow the expert cache

Evidence
  • T15 is not runnable: the offload engine's HostExpertStore only supports 3-bit experts (offload_store.py: if bits != 3: raise NotImplementedError), so 2.05 and 4.05 bpw crash at load in both images. Only 3.05 bpw remains (the CPU-expert fallback, SGLANG_EXL3_MOE_OFFLOAD=cpu, is far slower and not pursued).
  • T13 (PR #144 launch, f77c72f6, x8, boost off): lab C1 54.5–55.6, lab prefill 2,433–2,596, kit C1 48.2 (32k: 49.9), kit panel in band (top-1 0.9887, KL 0.00097), agentic whole-run 44.1, min 60-s window 37.9 (floor 50 fails).
  • Agentic per-request decode is flat across context (median 43.7 / 43.8 / 41.7 tok/s at <8k / 8–20k / 20–40k), so context length is not the limiter; the gap to lab C1 (55) is consistent with expert-cache misses on coding traffic (each miss streams experts over x8 PCIe). Hypothesis, to be tested by changing the cache size.
  • VRAM budget at ready (server log): expert cache 11.08 GB (auto-fit: free 19.82 − staging 0.48 − reserve 8.26), KV pool 5-bit 1.77 GB for 210k tokens, 3.23 GB still free after CUDA graphs. Context length is therefore a weak lever (the whole KV pool is 1.77 GB); the reserve is the strong one.
Grid (base = PR #144 launch on the latest main-built image a21efea6, GPU2 x8; one variable per step)
StepChangeExpected expert cache
R1SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB 8.26 → 6.0~13.3 GB (+20 %)
R2reserve → 5.0 (only if R1 is stable and gains ≥ 3 %)~14.3 GB (+29 %)
N1SGLANG_EXL3_NGRAM_TIER nvme → pinned (32.6 GB pinned RAM) on the best Rn-gram reads from RAM
G1--max-running-requests 1 --cuda-graph-max-bs-decode 1 (single-stream agentic) on the best, then re-fit the reservefrees graph memory

Metric per step: minimum rolling 60-s window and median per-request decode in the 10-min agentic run (the floor), with lab C1 and lab prefill checked for regressions. Every step runs lab,ttft,stab; the kit panel is re-run on the winner only (cache size does not change the math). Stop rule: 3 consecutive steps < 3 % on the floor metric. If no step lifts the minimum window to ≥ 50, Phase 1 closes with "not reached at x8", publishing the best config.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Cache misses are a hypothesis: the floor needs +31.9 % (26.4 → 20.0 ms/token); a 20–29 % larger cache need not deliver that. Expert-cache hit/miss and transfer stats are recorded if the engine logs them, else marked unavailable.
  • Order: R0 (unchanged PR #144 launch on a21efea6) → R1 reserve 6.0 → G1 single-stream graphs on R1 → reserve refit only if peak VRAM (prefill and max-occupancy) shows room → N1 pinned n-grams with host-RAM headroom recorded. R2 (5.0) is not run blind: measured headroom after R1 would be ~0.97 GB, after R2 ~−0.03 GB.
  • max-running-requests 1 is a recipe concurrency constraint, stated as such.
  • Floor = minimum rolling 60-s generation-only window after the first 60 s (harness definition); window coverage reported. One 10-min run screens; the winner then needs full §5 qualification, kit band, and ≥ 128k occupied for "agentic-usable". Closure wording: "50 tok/s floor not reached in the tested x8 configurations".
R0 result and re-plan (2026-10-04 22:42 AEST)

R0 (a21efea6, PR #144 launch unchanged): load 260 s, lab C1 55.8, lab prefill 2,419, kit C1 47.4 (32k 48.6), TTFT 2,009 / 2,681, panel in band (top-1 0.9897, KL 0.00097) — then the 10-min agentic run crashed: CUDA OOM. After serving starts, Triton kernels load lazily (log: "device-loaded after serving started … free 0.83 → 0.09 GiB") and consume the 3.23 GB that looked free at ready. So the 8.26 GB reserve is needed on this image; shrinking it cannot grow the expert cache safely. (f77c72f6 survived the same agentic run at the same reserve.) Re-plan: R1 (6.0) is already running and is kept as evidence; R1+G1 dropped. Next: R0 repeat (is the OOM reproducible?), G1 at reserve 8.26 (single-stream frees graph and request memory → headroom), N1 pinned n-grams.

  • R1 (reserve 6.0, 22:42): load OK, then CUDA OOM at the first lab request — reserve cannot go below 8.26.
  • R0 repeat, agentic run only (22:58–23:22): no OOM; 71 tool calls, 0 parse failures, whole-run 43.3, min window 39.7; lab after 57.4 / 2,537. The first R0's OOM is intermittent (it followed lab, kit sweep, TTFT and panel on the same server). Reported as an a21efea6 stability risk for PR #144; frequency unknown from 2 runs.
  • G1 single-stream at reserve 8.26 (23:22): min window 40.1 (+1 % vs R0 repeat), whole-run 43.1, lab 54.6–55.6.
  • N1 pinned n-grams (23:48): min window 40.9 (+3.1 %), whole-run 44.6, lab C1 53.7 / 51.3 (lower), prefill unchanged.
Closure (2026-10-05 00:11 AEST)

Stop rule met: R1 no gain (OOM), G1 +1 %, N1 +3 % (within run-to-run spread: two R0 runs and T13 span 37.9–39.7). Phase 1 result: the 50 tok/s floor was not reached in the tested x8 configurations. Best measured agentic floor ~40–41 tok/s (−18 to −20 %); lab C1 55–57, lab prefill ~2,400–2,600, kit panel in band. Best config for publication: the PR #144 launch (image a21efea6, or f77c72f6 which had no OOM in its run). No T16 finalist; T18 registry recipe not prepared; the PR #144 rerun evidence and the kit submission (with this tuning table) are prepared locally for Phase 3.

0011-flashnext-x8-tuning-grid.review.md 0011-flashnext-x8-tuning-grid.review
  • AMEND — growing the cache is a reasonable experiment, but the evidence does not establish cache misses as the bottleneck. Flat decode across context only weakens the context hypothesis. The floor needs +31.9% (37.9 → 50 tok/s); a 20–29% larger cache does not imply that speedup. The strongest counter-argument is that coding traffic accesses a broad expert set, so extra capacity may barely reduce misses.
  • Establish R0 on the new image first. T13 used f77c72f6; the grid uses a21efea6. Comparing directly changes both engine and reserve. Run the unchanged launch on the new, full digest-pinned image, then compare every tuning step against a baseline on that image.
  • R2 appears to exceed the measured memory budget. Assuming unchanged allocations and consistent units, R1 consumes another 2.26 GB, leaving approximately 0.97 GB after graphs; R2 consumes 3.26 GB, leaving approximately −0.03 GB. “Stable R1” cannot establish that R2 fits. Record peak VRAM during prefill and the claimed occupied-context run before reducing the reserve further.
  • Concrete alternative: R0 → guarded R1 → G1 → reserve refit → N1. Move graph reduction ahead of R2 when memory is tight, and measure each change separately. Treat max-running-requests=1 as an explicit recipe concurrency constraint; verify that the representative agentic workload fits it. For pinned n-grams, record host-memory headroom and end-to-end performance.
  • Test the mechanism alongside speed. Record expert-cache hit/miss counts, transferred bytes and transfer stalls if exposed; otherwise mark them unavailable and keep the attribution hypothetical. Going from 37.9 to 50 requires reducing generation time from 26.4 to 20.0 ms/token—a 6.4 ms/token saving. Establish whether expert transfers plausibly account for that much time.
  • Specify the floor calculation exactly. Ignore the first 60 seconds and calculate rolling windows using generation time only, excluding prefill, tools and idle time. Report window coverage and generation duration. Median request speed and whole-run average are diagnostics; neither establishes the floor.
  • Separate screening from acceptance. One 10-minute run can select candidates. The winner still needs the three §5 qualification runs—including fully cold and claimed maximum occupied context—≥100 tool calls across its runs, zero parse failures, repetition checks, all six lab gates afterward, and the kit quality band. “Agentic-usable” additionally requires ≥128k occupied tokens.
  • Narrow the closure claim. Close under the phase timebox or the specified three consecutive measured sub-3% gains, recording skipped branches explicitly. Publish “50 tok/s floor not reached in the tested x8 configurations” rather than implying x8 is exhausted. Prepare results and submissions locally; public publication remains Phase 3.
0012-glm-outcome.md 0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)

0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)

Result: no GLM-5.3-Flash configuration on this box reaches the registry's 15 tok/s gate; every route is a documented experiment, and no GLM registry PR is prepared. All numbers: lab C1 decode / lab prefill, current layout.

RouteCardsDecodePrefillToolsClassification
llama.cpp b11312 IQ2_M, ub409619.6288pass, 0 parse failuresexperiment (1 card < 15)
llama.cpp IQ2_M, ub2048, GPU2-first311.8218passexperiment (3-card, < 15)
llama.cpp IQ3_XXS / IQ4_XS310.4 / 8.3132 / 156passrejected for speed, quality unmeasured
llama.cpp Q2_K / IQ3_XXS19.2 / 8.378 / 65passrejected for speed
ik_llama 5bf8f0fe IQ2_M (no MTP)313.4–14.0185fails (tool calls unparsed, 0005)experiment-only, unattested image
ik_llama IQ2_M + MTP1 / 38.2–12.6~150–166failsexperiment-only; OOM / server crash
0xSero glm53-flash-offload, exact mode (x8)17.4–7.5720–739pass, 0 parse failuresquality reference; route below gate at x8
  • T25a: no current-layout finalist (decision 0007; rolling-floor failures measured in every 10-min run).
  • T24: the uncapped candidate-vs-reference comparison runs only for finalists, so it is not run. Exact mode loads on this box (~120 GB host RAM, 23.3 GB VRAM) and stays available as the reference if a future config qualifies.
  • T25b: 1-card routes < 15 → experiment; 3-card routes < 15 → no separate recipe; ik_llama → experiment-only until a tool-parser fix and an attested local-ai-images build (operator to raise with 0xSero).
  • T26: not applicable.
  • Hardware context for the reader: dual-channel DDR4-3200 (~16.6 GB/s host memcpy measured), x8 PCIe (13.5 GB/s), 15 cores at base clock (decisions 0004, 0010). Fast mode (0xSero's 28.9 tok/s route) needs ~238 GB RAM: not possible on AM4.
0013-flashnext-oom-fix.md 0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)

0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)

Operator request (2026-10-05): rerun the Qwen3.8 (Flash-Next) test and fix the stability issues.

Diagnosis (R0 first run, 2026-10-04 12:32 UTC)
  • The server died in a prefill: offload_runtime._prefill_parts_compact → `torch.AcceleratorError: CUDA error: unknown error, 29 s after Triton kernel '_row_logsumexp_kernel' device-loaded after serving started (free device mem: 0.00 GiB)`. Earlier in the same lifetime, during the 166k-token context gate, PyTorch logged repeated expandable_segments: memory mapping failed with OOM (free 10 MB of 25.3 GB).
  • At ready the engine reports 3.23 GB free, but kernels that load lazily after serving starts, plus prefill activations for 12,288-token chunks, consume it. Whether a run crashes depends on which kernels load late and the size of the next prefill: 1 of 3 lifetimes of the unchanged launch crashed; reserve 6.0 crashed at once.
Plan (GPU2 x8, full flow lab → kit sweep → TTFT → panel → 10-min agentic, the sequence that crashed)
  • Every flow now samples VRAM every 0.5 s (logs/vram/*.csv, summary = minimum free MiB): headroom is measured directly instead of waiting for random crashes.
  • Runs: baseline (unchanged launch) → A reserve 8.26 → 9.26 GB → B --chunked-prefill-size 12288 → 8192.
  • Pick the fix with the larger minimum free VRAM and the smaller speed cost (lab C1, kit C1, prefill, agentic floor), then repeat it twice (3 clean lifetimes) before calling it fixed. If neither leaves ≥ 1 GB minimum free, combine them.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Framed as memory-pressure mitigation, not proof the intermittent failure is gone: 3 clean lifetimes would still occur with probability 29.6 % at the old 1-in-3 crash rate (95 % upper bound on crash probability 63 %). The main evidence is the measured VRAM headroom margin; the diagnosis stays a hypothesis (the final unknown error is not itself a proven OOM). Allocator peak/reserved stats are not exposed outside the engine → recorded as unavailable.
  • Configs, one variable each: baseline (reserve 8.26, chunk 12,288), A (9.26, 12,288), B (8.26, 8,192); (9.26, 8,192) only if neither single change leaves margin. Actual expert-cache GB recorded from the server log for each.
  • Selection: Flash-Next has no qualifying config at x8 (floor not reached, decision 0011), so the target-based rule cannot pick a winner; the rule is **the config with the largest minimum free VRAM whose lab C1, kit C1, lab prefill and agentic floor stay within 3 % of baseline**; ties → baseline settings. ≥ 1 GB minimum free is an engineering margin, not a spec criterion.
  • Image provenance: a21efea6 = ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163, local-ai-images run 37113268282 on main, trellis-serve b1e74af; it replaces the PR's branch-built digest per 0xSero's PR #144 comment of 2026-10-03.
  • Note: the queue skipped the baseline line (identical text already in campaign.done from 2026-10-04); re-added with an explicit step list, so it runs after A and B.
Results (2026-10-05, GPU2 x8, same full sequence lab → kit sweep → TTFT → panel → 10-min agentic)
ConfigMin free VRAM (0.5 s samples)Lab C1Kit C1Lab prefillAgentic floor / avgOutcome
baseline (8.26, 12,288)2 MiB (10:41, kit sweep)49.947.82,419— / 43.0crash in agentic run, 6 tool calls
A (9.26, 12,288)912 MiB48.1–49.743.72,408–2,56437.4 / 41.1clean, 71 tool calls
B (8.26, 8,192)1,060 MiB49.5–54.948.22,131–2,23739.5 / 43.9clean, 64 tool calls

Baseline crash: Triton kernels loaded after serving started at "free device mem 0.00 GiB" (alloc_extend_kernel, _row_logsumexp_kernel, _sparse_gqa_chunk_prefill), then CUBLAS_STATUS_INTERNAL_ERROR in cublasGemmEx (cuBLAS could not get workspace). The unchanged launch has now crashed in 2 of 4 lifetimes; it runs the card to ~0 free VRAM. Chosen: B — the most headroom, decode unchanged (kit C1 +1 %, floor −0.5 %), prefill −8 to −12 % (still ~2,100+, over twice the 1,000 target). A costs 8–13 % decode (smaller expert cache). Two more B lifetimes queued.

Confirmation (2026-10-05 10:59–12:29 AEST): fix B, 3 lifetimes
LifetimeStep orderMin free VRAMLab C1 (before / after)Kit C1Agentic floor / avg, tool callsOutcome
B1lab, kit, ttft, panel, stab1,060 MiB54.9 / 49.548.239.5 / 43.9, 64clean
B2lab, kit, ttft, panel, stab1,060 MiB56.4 / 53.547.038.8 / 43.2, 62clean
B3lab, ttft, panel, kit, stab1,060 MiB55.0 / 45.048.140.4 / 44.0, 61clean

Result: memory-pressure mitigation confirmed for this sequence. The minimum free VRAM is identical in all three lifetimes (1,060 MiB), so the peak is deterministic and the margin is real, against ~0 MiB and a 2-in-4 crash rate for the unchanged launch. 187 tool calls, 0 parse failures across B1–B3. Cost: lab prefill −8 to −12 % (2,131–2,240). Lab C1 varies 45–56 between 30-s windows on every config (single-sample noise), kit C1 is stable at 47–48. Recommendation for PR #144 / the registry launch on 24 GB cards: --chunked-prefill-size 8192. The agentic 50 tok/s floor is a separate, unchanged result (decision 0011): ~39–40 tok/s at x8.

0013-flashnext-oom-fix.review.md 0013-flashnext-oom-fix.review
  • AMEND. The two interventions are reasonable, but the plan can declare “fixed” without demonstrating either the cause or §5 qualification. Treat this as mitigation testing; select the fastest qualifying config, with headroom as supporting evidence.
  • Strongest counter-argument: three clean lifetimes provide weak evidence of reliability. If the true crash probability remains 1/3, three successes still occur with probability 29.6%. With zero failures in three independent trials, the one-sided 95% upper bound on crash probability is 63.2%. Three runs satisfy the spec’s qualification count only when they meet its prescribed conditions; they do not establish that intermittent failure is eliminated.
  • The diagnosis should remain a hypothesis. Mapping OOMs and exhausted VRAM strongly support memory pressure, but CUDA error: unknown error does not establish the final failure’s cause. A kernel-loading message 29 seconds earlier is correlation. Record the failing allocation/request and available allocator diagnostics; describe the outcome as “memory-pressure mitigation qualified” unless causal evidence improves.
  • A 0.5-second VRAM sample is insufficient to certify peak headroom. It can miss allocation bursts, and device-free memory alone cannot distinguish reusable allocator reservations from live allocations. Supplement sampling with per-stage allocator peak allocated/reserved memory, allocation-failure records, and request occupancy/chunk size. Keep collection outside the engine and write logs to tmpfs as §9 requires.
  • Make the comparisons unambiguous. Use baseline (reserve 8.26, chunk 12,288), A (9.26, 12,288), and B (8.26, 8,192). Explicitly reset reserve for B. Test (9.26, 8,192) afterward if necessary, comparing it against either single-change config. This preserves §10’s one-variable-family rule.
  • Neither adjustment has a guaranteed memory benefit of the stated size. Raising reserve by 1.00 GB may reduce expert-cache capacity; reducing chunk size by 33.3% reduces only chunk-dependent allocations, not total peak usage. Standardize GB/GiB/MiB, record actual cache capacity, and justify the ≥1 GB threshold against observed peaks. It is an engineering margin, not a spec acceptance criterion.
  • Replace “larger minimum free VRAM and smaller speed cost” with an explicit selection rule. Require lab decode ≥50 tok/s, lab prefill >1000 tok/s, the quality band, and §5 on the same config at the claimed occupancy. Then choose the fastest qualifying config within the search budget. The cited 53.2 tok/s reference permits only about 6.0% decode loss before missing the target; this machine’s measured baseline determines its actual margin.
  • Make the three finalist runs conform explicitly to §5: cold with all specified caches reset, claimed maximum occupied context, and repeat; zero OOM/crash/parse failures, ≥100 tool calls across the config’s runs, repetition checks, the rolling decode floor, and all six lab gates after each run. A 166k context gate alone does not demonstrate agentic qualification there.
  • Resolve provenance before running. a21efea6 is insufficient identification, and it differs from the PR #144 digest specified in Phase 1. Record the full digest, source commit, attestation/allowlist eligibility, and reason for replacing the specified image. Store each immutable launch before execution and publish every failed attempt with explicit unavailable metrics.
0014-flashnext-vs-27b.md 0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)

0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)

Question: on this box, which is the better local model for agentic coding (and generally): Flash-Next (180B MoE, 6B active, experts streamed from RAM) or the dense 27B the operator runs daily?

Configs (both on GPU2, PCIe 4.0 x8, 15 cores, boost off; one card each)
  • Flash-Next: PR #144 Phase D launch on a21efea6 with the stability fix (--chunked-prefill-size 8192, decision 0013), EXL3 3.05 bpw, 204,800 context, up to 4 concurrent requests.
  • 27B: the operator's daily server exactly (omarchy-local-ai recipe qwen38-27b-exl3-3bpw-rtx3090-sglang-tp1): ghcr.io/0xsero/sglang-exl3@sha256:84f75f34…, EXL3 3.00 bpw (turboderp/Qwen3.8-27B-exl3@6fe61ad6…), NEXTN/MTP speculative decoding (3 steps, 4 draft tokens), 204,800 context, single stream.
  • Each is the best known recipe for its model on a 24 GB 3090; they differ in more than one variable by design (this compares models as deployed, not an isolated knob). Sampling, task set, token budget and harness are identical.
Measurements (same order for both; one server lifetime each)
AreaStepMetric
Speedlab (registry gates), kit sweep, ttftdecode C1 (C2–C4 where supported), prefill 8k–64k, TTFT
Long contextneedle (kit method, thinking off)retrieval at 128k (depths 0.2/0.5/0.8) and ~192k (0.5)
Agentic reliabilitystab (10-min agentic, 8-task §5 set)tool calls, parse failures, loops, decode floor
Agentic codingqeval (T07, uncapped, one pass) on tasks-compare30solve rate (official SWE-bench eval), tokens and wall time per task
Footprintload + VRAM samplerload time, host RAM, VRAM, min free VRAM

tasks-compare30: 30 SWE-bench-Lite test instances drawn with random.Random(20261005).sample (uniform, not hand-picked), images pinned by digest after pull; same 60k completion-token budget per task for both models. With n = 30, a difference in solve rate is reported with a paired comparison (tasks solved by one and not the other) and an exact McNemar p-value; small differences are labelled inconclusive rather than ranked.

Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Labelled an exploratory deployment comparison on this box (the §10 one-variable rule is waived for it, as the two recipes differ in engine build, MTP and concurrency by design). Conclusions are per priority — agentic coding success, interactive speed, long context, footprint — never "generally better".
  • compare30 is exploratory: exact McNemar on 30 single attempts needs ≥ 6–0 discordant tasks for p < 0.05 (8–2, a 20-point gap, gives p ≈ 0.11). The site publishes the paired per-task table, the solve-rate difference with a 95 % CI and the p-value; small differences stay unranked.
  • Cache state: the flow serves cold (container restart, page cache dropped) and then measures in a fixed order, so qeval runs warm for both; the 10-min stab run is screening, not §5 qualification. Flash-Next's kit band is already in band for this config (decision 0013: top-1 0.989, KL 0.00092); the 27B has no kit panel (different model), so its quality evidence is the solve rate only.
  • Budget: up to 60k completion tokens per task (all of a task's model calls) → worst case 1.8M tokens per model; early submits cut this sharply. Infrastructure failures (image, evaluator) are reported separately from unsolved. Accepted tokens/s and acceptance rate are recorded for the 27B's MTP.
Finding during the 27B run (2026-10-05 18:07 AEST)

The operator's 27B server advertises --context-length 204800, but its KV pool is max_total_num_tokens=135424 (mem-fraction 0.80, fp8 KV, MTP draft buffers). The registry lab's context gate (0.85 × 204,800 = 166,667 tokens) is rejected: "Input length (166667 tokens) exceeds the maximum allowed length (135418 tokens)". So the 27B as deployed really holds ~135k tokens, not 204,800 — the same advertised-vs-real gap seen in the operator's pi session. The comparison reports both; an extra lab run uses the identical server with harness context 131,072 (…-omarchy-ctx131k launch) so its speed gates are measured. Flash-Next's KV pool is 209,984 tokens (fits 204,800).

0014-flashnext-vs-27b.review.md 0014-flashnext-vs-27b.review
  • AMEND. Comparing the best deployed recipes is useful, and differing engines/MTP settings are legitimate for that question. The strongest objection is inferential: 30 tasks, one attempt each, and one server lifetime cannot reliably establish which model is better “generally.” Frame this as a deployment comparison on this machine, with separate conclusions for coding quality, latency, throughput and resource use.
  • Thirty tasks can miss substantial differences. With exact two-sided McNemar, 5 wins versus 0 losses gives p = 0.0625; 6 versus 0 gives p = 0.03125. Even 8 versus 2—a 20 percentage-point solve-rate advantage—gives p ≈ 0.1094. Publish the paired outcome table and a confidence interval for the difference. Keep compare30 exploratory; if a definitive ranking matters, predeclare a larger fixed sample before examining results.
  • One stab run is screening, not §5 qualification. Unless this exact Flash-Next config already has qualifying evidence, require the three qualification runs: cold, claimed maximum occupied context, and repeat; ≥100 aggregate tool calls; the rolling decode-floor measurement; and post-run lab gates. Otherwise label it experiment-only and its stability unqualified. “One server lifetime” needs an explicit exception for the required cold restart.
  • The measurement order systematically warms Flash-Next’s streamed experts. lab → kit → ttft → needle → stab → qeval measures coding after extensive preparation, whereas cold agent startup may behave differently. Keep the warm deployment comparison, but add the specified cold qualification and cold first-task latency. Disable prefix caching and use fresh prompts for TTFT; record expert, n-gram and OS cache states separately.
  • Configured context and needle retrieval do not establish agentic usability. A ~192k needle pass supports retrieval at that occupancy, not reliable coding there. Report server-counted occupied tokens and ≥4k generation headroom. Award “agentic-usable” only after §5 qualification at ≥128k occupied tokens; otherwise say “configured for 204,800.”
  • Restore the explicit Flash-Next quality gate. Require or cite matching evidence for top-1 ≥0.987 and KL ≤0.0013 against the pinned reference panel before any spec headline or registry claim. SWE-bench solve rate complements that requirement; it does not replace it. Publish outside-band results with the prescribed label.
  • Make the deployment exception reproducible. Explicitly amend §10’s “one variable family” rule for this comparison and avoid attributing differences to architecture alone. Store full image digests, 40-character weight revisions, file hashes, immutable launches, GPU UUID, harness/evaluator commits, ordered task IDs and the source dataset revision. Record accepted tokens/s, MTP acceptance rate and draft settings as §4 requires.
  • Account for cost and stochastic variation. The completion allowance totals 3.6 million tokens across both models—20 hours of generation at 50 tok/s, before prefill, tools and evaluation. Reserve that time within the phase budget. Define which tokens count toward 60k, distinguish infrastructure failures from model failures, and report total wall time and tokens per successful solve alongside per-task results. A single attempt measures this run’s outcomes; paired statistics do not eliminate sampling variability.
  • Concrete alternative: run compare30 as a clearly labelled exploratory deployment comparison; reuse documented qualification evidence where it exactly matches, complete missing qualification/cache checks, and publish a tradeoff table. Recommend a model only for a stated priority—such as coding success or interactive latency. Remove “and generally,” and leave small or uncertain quality differences unranked.
0015-next-tests-plan.md 0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)

0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)

What the thread says
  • Operator showed local-ai-bench.org/compare and the SWE-bench-Lite contamination finding; asked "Shall I create a benchmark from recent github issues?" — Sero: "Yes". Operator: "land a novel benchmark, and make it real", and later "an agent/coding benchmark for multiple languages without disclosing the questions/content".
  • Sero (funding): keep running experiments, turn them into cool demos (Omarchy + the local-ai plugin, "wacky stuff") that he will amplify; fundraiser in a few weeks with 3 goals (small/medium/big). Hardware advice: stick with NVIDIA (FLOPs, CUDA) — vs the operator's ideas (Epyc 256 GB + 4th 3090, or an M5 Ultra).
  • Operator wants to go deep on Qwen3.8-Flash-Next ("Qwen 4 is around the corner").
  • Links: Niko1221/Strata (operator: "amazing decoding numbers … how much better than Qwen 3.8 27B?"); kacper-daftcode/vLLM-Moet + 0xSero/moetier (Sero: "give this to claude your rig will fly", "I don't use gguf"; operator: vLLM-Moet says Blackwell → Sero: moetier "shouldn't need Blackwell"). Operator: "Will test these through the benchmark."
  • Repo facts: Strata = C++ engine for Flash-Next GGUF quants (Q2_0 … IQ3_S, "Coder" IQ1_M); claims 94 tok/s on a 12 GB RTX 5070 and "100–140 tok/s" expected on a 3090. vLLM-Moet = hand-written SM120 (Blackwell) SASS kernels — cannot run on SM86 3090s. moetier = Sero's data-first standard + simulator for expert placement across VRAM/RAM/NVMe; its loop's primary target is GLM-5.3-Flash on 1× 3090 + 55 GB DDR4 + NVMe at 50 tok/s, with run records (registry/runs) carrying a standard table, utilization and chokepoints (moetier probe).
Plan (ordered; each item ends in published records)
  1. Tell Sero the PR #144 stability finding now (operator sends; draft in prs/pr144-proof.md): a21efea6 runs a 24 GB card to 2 MiB free and crashed 2/4; --chunked-prefill-size 8192 fixes it (3/3, decode unchanged). He is the maintainer, it is the cheapest high-value result we have.
  2. moetier on this box (Sero's explicit ask). (a) hardware record for omarchy-gpu (GPU2 x8 13.5 GB/s, 2-ch DDR4, 16.6 GB/s memcpy, 1 TB NVMe 3.4 GB/s rand) and a Flash-Next model record if missing; (b) moetier probe on the trellis Flash-Next config → utilization + chokepoints run record: name the binding constraint behind the ~40 tok/s agentic floor at x8; (c) reproduce his GLM-5.3 NVMe-mode recipe at a 55 GB RAM budget on our single NVMe (an independent hardware datapoint for his G1/G2 work); (d) moetier sim the fundraiser hardware goals.
  3. Strata vs 27B vs trellis Flash-Next (operator's question). Pinned Strata release; Flash-Next GGUF quants (Q2_0, IQ2_XS, IQ3_S, Coder) on GPU2, then 2-3 cards GPU2-first. Same suite as decision 0014 (lab, kit sweep, TTFT, needle, 10-min agentic) + quality vs the EXL3 3.05 reference (KLD/top-1 where the API exposes logprobs). Test the "100–140 tok/s on a 3090" claim. GGUF → operator-facing comparison, not a registry recipe (Sero).
  4. Fresh-issues benchmark (decontaminated, held out). Pilot: ~50 Python tasks from issue-linked PRs merged after 2026-09-01 (after both models' release), auto-built Docker envs (reuse SWE-rebench / SWE-bench-Live RepoLaunch tooling), validated FAIL_TO_PASS/PASS_TO_PASS with the gold patch, run through our harness. Held out: publish aggregate scores, repo list, task-ID hashes and a few retired samples; refresh monthly. Then add TypeScript, Go, Rust (multi-language, per the operator). Keep the gold-overlap check as a standing guard.
  5. Run the three contenders on it: 27B, trellis Flash-Next (fixed), best Strata quant; paired analysis.
  6. Demos for Sero / the fundraiser: Omarchy + local-ai plugin — (a) 27B as a fast coding agent (one-shot app), (b) Flash-Next answering over a ~190k-token codebase; short screen recordings + numbers from the records.
  7. Fundraiser evidence: per goal (small: 4th 3090 / 2nd NVMe; medium: Epyc 8-ch 256 GB + x16 lanes + 4th 3090; big: a Blackwell card → vLLM-Moet/moetier), projected tok/s from moetier sim calibrated on our measured runs.
  8. Qwen 4 day-0 readiness: launch templates + the fresh-issues set ready so a new model runs the same day.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Milestone 0 first: Phases 1–2 are closed as documented results (0011, 0012); their Phase 3 material (PR #144 evidence, kit submission) goes out before new work is presented as findings. PR #144 wording: "no crashes in 3 full-sequence runs with chunked prefill 8192 (vs 2 of 4 without); decode unchanged within ±2 % (kit C1 47–48); prefill −8 to −12 %" — a mitigation, not a proven fix.
  • moetier: answer one measurable question — what binds the ~40 tok/s agentic floor (bytes per token over PCIe, expert-cache misses, CPU lane, GPU idle) — with measured counters, then change one variable family. The GLM reproduction states its RAM budget in GiB including pinned memory, swap off, cache resets declared; 50 tok/s is Sero's hypothesis, and registry gates still apply.
  • Strata: screen quants on GPU2 only (one 10-min agentic run + speed suite each), qualify only finalists, and add 2–3 cards only for a finalist with a measured reason. Verify what "Coder" changes (weights/behaviour). Quality without logprobs = "unavailable", never in-band.
  • Fresh-issues set = "temporally held out", not "decontaminated": require issue and fix after 2026-09-01, overlap check against existing benchmarks, no network access for the agent during evaluation, tasks frozen before any tuning. A separate small development set is used for choosing configs; finalists run once on the held-out set. 50 tasks is a pilot (±14 pp at 50 %); report paired, repository-aware uncertainty.
  • Secrecy vs reproducibility: a private immutable manifest (commits, patches, tests, env digests, harness settings) with a published commitment hash; controlled reproduction on request; full artifacts released when a task set retires. Active-set raw logs stay out of the public site (SPEC §8 amended accordingly).
  • Fundraiser numbers are conditional projections with stated assumptions and sensitivity ranges, kept apart from measured results; topology (slots, lanes) is checked before any 4th-3090 projection.
0015-next-tests-plan.review.md 0015-next-tests-plan.review
  • AMEND. The strongest objection is scope drift: the agreed deliverable is qualified Flash-Next and single-3090 GLM recipes; this plan adds a benchmark product, several engine comparisons, demos and hardware projections before closing those deliverables. Keep the new work, but put it behind an explicit milestone: publish the fastest qualifying configs—or documented failures within the spec’s timebox—and prepare their submissions.
  • Report PR #144 as a promising mitigation, not a proven fix. Moving from 2/4 crashes to 0/3 is useful evidence, but three successes still permit a per-run failure probability of 63% at a one-sided 95% confidence bound under independent, identical trials. Use “no crashes in three qualification runs,” verify the same occupied context and workloads, and quantify “decode unchanged.” Confirm Phase 3 publication requirements are already satisfied or explicitly amend them.
  • Make moetier answer a measurable bottleneck question. Going from ~40 to 50 tok/s requires 25% more throughput, or 20% less time per token. If PCIe alone were saturated at 13.5 GB/s, that implies reducing traffic from approximately 338 to 270 MB/token. These are conditional bounds, not a diagnosis. Measure transferred bytes, expert-cache misses, CPU work and GPU idle time, then change one variable family. A utilization report alone cannot establish causality.
  • Define the GLM reproduction before running it. Specify whether “55 GB” means decimal GB or GiB, and whether it includes pinned allocations, page cache and engine overhead; declare swap and cache-reset behavior. Treat 50 tok/s as the supplied recipe’s hypothesis, not an expectation. Registry eligibility still requires the ≥15 tok/s floor, uncapped quality evaluation and ≥80% of reference solves; reproducing a fast simulator or engine run does not satisfy those gates.
  • Separate Strata’s speed claim from its eligibility. The supplied 100–140 tok/s claim is 2–2.8× the Flash-Next target, but needs matching context, cache state, sampling and accepted-token accounting. A low-bit quant can win throughput and fail quality. Missing logprobs mean “quality unavailable,” which cannot establish the required top-1 ≥0.987 / KL ≤0.0013 band. Verify whether “Coder” changes weights or model behavior before presenting it as a quant-only comparison.
  • Rename the fresh set “temporally held out”; “decontaminated” overclaims. A PR merged after model release can fix an issue published long before release, and an agent with network access can retrieve the public solution. Require fresh issue/test provenance, inspect overlap with existing benchmarks, block solution retrieval during evaluation, and freeze evaluation tasks before tuning. These controls reduce contamination risk; they do not prove its absence.
  • Treat 50 tasks as a pilot, not a decisive leaderboard. At a 50% solve rate, 50 independent tasks give an approximate 95% margin of ±14 percentage points; one task changes the score by 2 points. Repository clustering can make uncertainty larger. Report paired outcomes and uncertainty based on discordant tasks, with repository-aware analysis. Use a development set for choosing the “best Strata quant,” then evaluate finalists once on the held-out set.
  • Resolve secrecy versus reproducibility explicitly. Public aggregate scores, repository names and task-ID hashes do not let another evaluator reproduce scores, and hashes of enumerable public IDs may reveal tasks. Keep a private, immutable manifest containing base commits, patches, tests, environments and harness settings; publish its commitment and arrange controlled independent reproduction. Amend §8 so public raw logs cannot accidentally expose active tasks, with full artifacts released upon retirement.
  • Make fundraiser projections conditional, with uncertainty. A fourth 3090 adds 24 GB nominal VRAM, not guaranteed usable bandwidth or speed; the actual slot, lane and chipset topology must be established first. A second NVMe helps only if measured storage contention is binding. EPYC and Blackwell projections require calibration beyond this AM4/SM86 machine. Publish assumptions and sensitivity ranges; keep measured results separate from simulated forecasts.
  • Concrete replacement order: narrow PR #144 report → close existing Flash-Next/GLM qualification and submission material → probe the measured bottleneck → screen Strata on GPU2 and qualify only finalists → freeze and validate the 50-task pilot → paired contender evaluation → demos and conditional fundraiser scenarios. Four quants across three layouts already imply 12 configs: up to eight hours of screening plus qualification alone, before loading, lab, sweep and quality checks. Reserve multi-card expansion for a single-card finalist with a demonstrated reason to benefit; label the ~190k demo by actual occupancy and qualification status.
0016-fresh-issues-stage2.md 0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")

0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")

Context: decision 0015 item 3 (temporally held-out Python set, issue AND PR created ≥ 2026-09-01, private tasks, published scores + commitment hash). Stage 1 (tools/fresh/, data in ~/dev/local-ai-exp/fresh-private/, never in this repo) gave a pool of 5,450 candidates and a ≤ 3-per-repo shortlist of 763 (275 repos; 249 easy / 271 medium / 243 hard; 528 from repos not in any public SWE dataset). Operator choice (2026-10-08): the setup/filter LLM is this Claude subscription (claude -p, Opus) — not an API key; Anthropic therefore sees which tasks we chose ("semi-private for Anthropic models", stated next to every score).

Plan
  1. Stage-2 input (deterministic, before any LLM): from the shortlist drop rows with base commit dated before 2026-08-24 (5), no .py test file (5), and repos created after 2025-09-01 or with < 10 distinct contributors in the last year (guards against single-author agent-generated issue→PR streams seen in the top repos). Sample 150 by difficulty (easy 45 % / medium 40 % / hard 15 %), seen/unseen repos ≈ 1:2. Seed fixed, list recorded.
  2. Clarity filter (Opus, one claude -p call per task, JSON out): the SWE-bench-Verified rubric — is the issue well specified, can a correct fix be judged by the hidden tests without guessing names/messages that only the tests define, is the fix given away in the issue. Reject scores ≥ 2 (0–3 scale) on either axis.
  3. Environment (deterministic skeleton + agent-written setup):
    • Repo snapshot: git init + git fetch --no-tags origin <base_commit> (ancestors only; no branches, tags or later objects exist in the image), checkout detached, remote removed, reflog expired, gc --prune=now; a post-build check asserts git rev-list --all contains no commit dated after the base and the gold blobs are absent.
    • Base image python:<ver>-bookworm pinned by digest. Opus (claude -p --model opus, Bash tool restricted to docker exec into a throwaway per-task container with network, 45-min cap, ≤ 60 turns) writes setup.sh (system packages, pip/uv installs) and the test command pytest -rA <test files>; installs pinned to versions available before the PR's creation date where the agent can (uv pip --exclude-newer). The final image is rebuilt from setup.sh without the agent (reproducible), tagged lab/fresh.<id>, digest recorded.
    • LLM-written log parsers are not used: pytest -rA + swebench parse_log_pytest only (no host-side code from the agent).
  4. Validation (no LLM): in the image, run the test files on base + test_patch (expect failures), then base + test_patch + gold patch; FAIL_TO_PASS = fail→pass, PASS_TO_PASS = pass→pass. Require ≥ 1 F2P, gold resolved 3/3 (flaky tests dropped from P2P, task dropped if an F2P is flaky), empty patch resolved 0/1, test run < 10 min. Rows emitted in our swebench 5.x format (eval_script, parse_log_pytest, version fresh-2026-09, image digest) into a private task dir; swebench evaluates them with network off for the agent under test.
  5. Freeze: from validated tasks, re-apply ≤ 3 per repo, stratify, pick 50 test + 10 dev (dev is for harness debugging only; never reported). Manifest (ids, base, patches, digests) committed only in fresh-private; publish sha256(manifest || salt) and the counts/distribution now, the salt and manifest when the slice is retired.
  6. Runs: Qwen3.8-27B (ctx131k), Flash-Next trellis c8192, Strata finalist; same harness as compare30 (uncapped, one pass), paired analysis (McNemar), gold-overlap per task as a leak check (flag > 50 % verbatim reproduction).
Scheduling and cost

Stage 2 is CPU/disk heavy (docker builds, test runs): it runs only as sh lines of the campaign queue, never during a GPU measurement. Parallelism 3 tasks. Disk: ~1–2 GB per image → ≤ 200 GB worst case before pruning failed builds. Subscription usage: ~150 filter calls + ≤ 150 setup sessions; stop and report if rate-limited.

Risks

Yield may land < 50 (published Python validation yield ~19 %): widen to the pool (reserves) and to October issues as a second slice rather than relaxing the date rule. A repo snapshot without later commits can still contain the fix if it was committed to the base before the PR (rare; the gold-overlap check at run time flags it).

Outcome after review (GPT-6.1 Sol: AMEND) — accepted
  • Supplementary pilot, contamination-reduced (not "uncontaminated"); labels next to every score: "Opus-curated", "semi-private for Anthropic models", rubric + Opus id + rejection counts published. Does not replace §5 qualification or the reference panels. Public release only after Phase 3 review.
  • Ordered batches, not 150: an ordered September list (stage-2 filters applied, stratified, seed fixed) is frozen now; attempts run in batches of 50 until 60 validated (expect ~320–380 attempts). Budgets: setup cap 30 min/task, ≤ 3 parallel, stage-2 wall time ≤ 40 h, image disk ≤ 150 GB (rejected images pruned immediately). If a budget runs out, stop with an undersized pilot and say so. October = separate slice.
  • Git invariant: image HEAD == base SHA; refs = HEAD only; git rev-list --all == ancestry of base; no stray working-tree files vs git status; gold-patch files are never mounted or copied into setup containers. Base must fail the hidden F2P tests (acceptance check).
  • Reproducibility: keep setup.sh, resolved pip freeze/uv lock with hashes, apt package versions, build logs, test command and image digest. Setup containers: CPU/RAM/PID limits, no host secrets, no Docker socket.
  • Validation: collected node IDs must match between runs; skips/xfails/collection errors counted explicitly; test patch must not disable tests; each repetition in a fresh container; gold 3/3 + empty 0 recorded with every exclusion and reason (3/3 is weak evidence on flakiness — reported as such).
  • Protocol frozen before candidate runs: manifest bytes, high-entropy salt, ordered reserves, selection rules, harness/parser SHAs, sampling, per-task token budget (same for all models; the compare30 harness settings), tool budget, context policy, retries, timeout treatment. Weight revisions recorded per model.
  • Reporting: one primary comparison predeclared — Strata finalist vs Qwen3.8-27B (the operator's question); others exploratory. Discordant counts, exact McNemar, repo-clustered intervals, ±14 pp margin stated at n = 50.
  • Gold overlap: descriptive audit only (denominator = gold added lines), never used to exclude or rescore.
Predeclared before any candidate run (2026-10-09 05:50 AEST)
  • Strata finalist = IQ3_S (launch strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb): T41 screen agentic floor 87.3 tok/s (Q2_0 123.3), compare30 29/30 (Q2_0 28/30); chosen as the higher-quality quant that still clears the 50 tok/s floor. Primary comparison: Strata IQ3_S vs Qwen3.8-27B (ctx131k launch), exact McNemar on the 50-task test slice. Exploratory: Flash-Next trellis c8192, Strata Q2_0.
  • Harness: tools/agentic/quality_eval.py at the commit recorded in each result, uncapped one pass, 60,000 completion tokens per task, sampling temperature 0.7 / top_p 0.95 / seed 1234 / max_tokens 8192 (compare30 settings), agent container network none.
0017-site-structure.md 0017 — Site structure: story-led home, findings, recipes, model cards

0017 — Site structure: story-led home, findings, recipes, model cards

Operator request (2026-10-10): pages were messy; home must show the sequence of what was done and a summary of achievements; every model gets a model card (link, details, learnings to run). Debated with GPT-6 Astra (Codex, 2 rounds) before building.

Outcome
  • Navigation: Home · Findings (speed, coding quality, GLM-5.3-Flash, hardware) · Recipes · Models · Method & Hardware · Runs & Data · Funding & Contributions. Old tabs (#charts, #compare, #fresh, leaderboards) kept for links, marked superseded or moved under Data.
  • Home: dated current answer (speed config and task-success config named separately), three bounded achievement summaries with conditions/limits/evidence, the 10-step dated timeline (outcome-led, chips, decision links), contributions with real status, hardware hypotheses, next steps.
  • Editorial text lives in site/content.json (operator-reviewed); charts and numbers in tables are computed from records.
Proposal (Claude)
local-ai-bench.org — information architecture proposal (draft by Claude)
Context

Static site generated from ~250 result records (tools/site/build.py). Audience, in order: (1) 0xSero, maintainer of the local-ai registry/recipe-kit and our lead, (2) potential fundraiser backers (hardware goals small/medium/big), (3) local-LLM practitioners who want a recipe for 3090-class boxes. The operator says the current pages "look very messy between them".

Current tabs (10 + 247 run pages)

Overview (per-model status cards, PR list, recent runs, a new headline callout) · Flash-Next leaderboard · GLM leaderboard · 27B leaderboard · Fresh-issues benchmark (method + results mixed) · Flash-Next vs 27B (public SWE-bench-Lite comparison, memorised) · Charts (8 charts with no narrative) · Runs (247) · Hardware & methodology (hardware, rules, decisions, spec). Problems: the story is nowhere; results for one question are spread over 4 tabs; three near-identical leaderboards; charts detached from the claims they support; method and results mixed; the old compare page contradicts the new benchmark unless read with a caveat.

What was done (chronological, all dated AEST, each backed by records + a decision file)
  1. Oct 3–4 · Box baseline: 3x RTX 3090 on AM4 (GPU2 x8 13.5 GB/s, GPU0 behind chipset 6 GB/s), faulty CPU core found and quarantined, boost off after machine-check errors (decisions 0004, 0010). Card order matters: GPU2-first +43–51 % prefill (0006).
  2. Oct 4 · Flash-Next on trellis/SGLang (Sero's PR #144 launch): lab 55 tok/s, prefill 2,400, but agentic floor ~40 at x8 — 50 not reached in any tested x8 config (0011). x16 card move dropped by the operator (0009).
  3. Oct 4–5 · GLM-5.3-Flash on one 3090: every route < 15 tok/s (0012).
  4. Oct 5 · PR #144 image OOM found (2/4 crashes) and mitigated with chunked prefill 8192 (0013). Public site launched.
  5. Oct 5–6 · Flash-Next vs 27B on public SWE-bench-Lite: 30/30 vs 18/30 — but Flash-Next reproduces upstream fixes verbatim (memorised) (0014).
  6. Oct 8 · From the Sero thread (0015): PR #144 rerun comment + kit submission PR posted; Strata (C++ GGUF engine) screened: Q2_0 floor 123 tok/s, IQ3_S 87 tok/s on one x8 card; moetier on the box: GLM NVMe recipe 6.1 tok/s at 55 GiB → 11.3 at 90 GiB; probe shows NVMe-bound → CPU-bound; trellis Flash-Next is PCIe-bound.
  7. Oct 8–10 · Built a private fresh-issues benchmark (0016): 430 attempts → 50 tasks, issue+PR after 1 Sep 2026, commitment hash published. Strata IQ3_S 48/50 vs 27B 36/50 (exact McNemar p = 0.0018, predeclared).
Proposed structure (5 top-level sections)
  1. Home = the story. Hero: the current answer in one sentence + 3–4 achievement tiles (fastest agentic setup on one 3090; best coding quality on fresh tasks; GLM verdict; contributions upstream). Then the dated timeline above as a vertical sequence, each step with its one key number, a status chip (result / negative result / fix / tool) and a link to the finding. Ends with "What's next" (Qwen 4 readiness, multi-language slice, fundraiser hardware sims).
  2. Findings (one page per question, narrative + the charts that support it + links to runs): a. Flash-Next speed on one RTX 3090 (engines compared; why x8 PCIe caps trellis; Strata result; stability fix) b. Coding quality (fresh-issues results; the memorisation finding on public tasks; 27B vs Flash-Next) c. GLM-5.3 on one RTX 3090 (routes tried; moetier NVMe recipe; what limits it; what hardware would fix it) d. Hardware lessons (PCIe lanes and card order, faulty core, NVMe at x2, RAM caps)
  3. Methods: how we measure (lab gates, kit sweep, TTFT, stability floor, quality panel, probe) and the fresh-issues benchmark construction (moved out of the results page), hardware record, rules for statuses.
  4. Data: one filterable config table (model/engine/quant/cards filters, replaces the three leaderboards), all runs, run detail pages, decisions log.
  5. Upstream: PRs and submissions to Sero's repos with status, and recipes ready to copy (launch commands).

Charts tab removed: every chart moves into the finding it supports. The old "Flash-Next vs 27B" page becomes a section of Findings b, framed as the memorisation evidence.

Questions for the reviewer

Is this the right structure for these audiences? What would you cut, merge or reorder? Is a timeline the right home page, or should the home lead with findings? Anything that makes the site look less credible?

Round 1 (GPT-6 Astra)

The proposal fixes the fragmentation, but risks replacing ten messy tabs with a polished lab diary. Lead with findings and usable recipes; make the chronology secondary. A public visitor needs to know what works, under which conditions, and how strong the evidence is.

0xSero’s priorities deserve a prominent contribution summary, not control of the entire navigation. Practitioners need reproducible setups; backers need evidence that a specific purchase would resolve a measured constraint. Neither audience benefits from reading seven dated steps first.

I would use five sections, in this order:

  1. Results — question-led findings, with a comparison summary.
  2. Recipes — tested launch commands, requirements, expected performance, limitations.
  3. Method & Hardware — measurement definitions, benchmark construction, exact machine configurations.
  4. Runs & Data — filterable results, individual runs, downloadable records, decisions.
  5. Project & Funding — upstream contributions, dated progress log, hardware proposals.

Home remains accessible through the logo. “Upstream” is insider language and combines two different jobs: tracking contributions and getting software running. Recipes should not be buried there.

The home page should contain, in order:

  • A bounded answer: which tested configuration you currently recommend for which workload, on what hardware, as of what date.
  • Three finding cards: practical agent performance, fresh-task coding results, and GLM feasibility. Each needs a configuration, metric, limitation, and evidence link.
  • “Run a tested setup” linking directly to recipes.
  • “What additional hardware would test” showing small/medium/big funding options, the measured constraint each addresses, cost, and success criterion.
  • “Recent changes” with three entries and a link to the full timeline.

Cut the achievement tiles. “Fastest agentic setup on one 3090” is especially risky when the supporting numbers mix laboratory throughput, agentic floors, engines, and quantizations. “Best coding quality” overstates a two-configuration, 50-task comparison. Use “Highest measured [defined metric] among tested configurations” and “48/50 versus 36/50 on this task set.” Put contributions below results, not alongside them as equivalent scientific achievements.

Keep the question-led findings, but sharpen their titles:

  • Which setup runs coding agents fastest?
  • How often do the tested setups solve fresh issues?
  • Is GLM practical on this machine?
  • Which hardware constraints affect performance?

The largest credibility issue is comparability, not navigation. “One RTX 3090” obscures a three-card host with unusual lane allocation, CPU restrictions, RAM requirements, and NVMe constraints. Display active GPUs separately from host hardware. Every comparison should expose model, engine, quantization, workload, context length, and relevant resource settings.

The fresh benchmark needs a compact evidence box beside its headline—not only a distant Methods link. Explain how 430 attempts yielded 50 tasks, exclusions and their timing, whether selection involved the evaluated systems, paired outcomes, and what was predeclared. A commitment hash needs a timestamp and a clear account of what it commits to. Recent issue dates alone do not establish contamination freedom; the p-value does not establish general coding superiority.

Keep chart integration, consolidated data, negative results, and the decisions log. Preserve old URLs with conspicuous superseded-result notices. Describe verbatim fixes as evidence consistent with memorisation unless stronger provenance establishes it. Treat proposed hardware remedies as hypotheses until tested.

Round 2 (GPT-6 Astra, after the owner's constraint)

The owner’s constraint changes my recommendation: make chronology the home page’s main structure, with the current answer and achievement summary above it. I withdraw the blanket objection to tiles; bounded result summaries serve a useful purpose.

1. Home, top to bottom

  • Current answer. A short, dated statement distinguishing the fastest tested configuration from the configuration evaluated for task success. Explicitly separate Q2_0’s speed result from IQ3_S’s 48/50 result. Name the workload, active GPU count, and three-card host; link to the applicable recipes.
  • Three achievement summaries: agentic performance, fresh-task outcomes, and GLM feasibility. Each gets a metric, essential conditions, limitation, and evidence link. Keep them compact enough that the answer and summaries fit a typical desktop first screen; on mobile, prioritize the answer. Put the fresh-benchmark evidence box directly beside or beneath its summary, not behind a tooltip.
  • “How we reached this answer.” The dated, chronological sequence is the page’s main body: 6–8 outcome-led steps, each linking to its finding and underlying records. Show the initial constraint, subsequent result, and any change in interpretation. Include the public-benchmark result together with the memorisation caveat.
  • Contribution summary. Show the stability work, rerun report, and kit submission with accurate statuses and links.
  • “Run a tested setup.” Direct recipe links with requirements and expected results.
  • “What additional hardware would test.” Measured constraints, proposed purchases, costs, hypotheses, and success criteria.

2. Final top-level navigation

Home → Findings → Recipes → Method & Hardware → Runs & Data → Funding & Contributions

Home deserves an explicit label because the owner makes the sequence a primary destination. Combining funding and contributions keeps project support together without burying recipes.

3. Remaining critical objections

Your timeline revision works, but “one number per step” should be a preference, not a rule. Some outcomes require a denominator, before/after values, or paired conditions. Clarity matters more than visual uniformity.

The proposed tiles still need corrections:

  • “Strata Q2_0” identifies an engine and quantization, not the model. Include the model and define “floor”; ten minutes alone does not explain that metric.
  • “GLM-5.3” must preserve the exact tested variant, apparently GLM-5.3-Flash, and say “across tested routes on this host.”
  • “Predeclared” must identify what was declared and when. The p-value belongs with paired outcomes and selection disclosures.
  • “2 upstream contributions” must distinguish submitted, accepted, and merged work.

Most critically, never combine Q2_0 speed and IQ3_S task success into an implied single configuration’s achievement.

Budget and stop-rule log (budget.md)

Budget

PhaseStartedBudgetStatus
0 setup2026-10-021 dayin progress
1 Flash-Next–2–3 days–
2 GLM–2–3 days–
3 review & PRs–operator-paced–
Stop-rule log
DateRouteLast 3 gainsDecision

Site rules

Status badgenot started = no records; done = a config with a kit / registry / kit+registry / experiment-only eligibility whose stability verdict is qualified; otherwise running. site/status.json overrides it (shown as override).
Stability verdictfailed = any stability run failed or crashed, or shows parse failures, repetition hits or crash/OOM; qualified = at least 3 passing runs, one of them cold, and at least 100 tool calls across them; screened = passing runs short of that.
Flash-Next targetone passing lab record of the config with decode >= 50 tok/s and lab prefill > 1000 tok/s. Headline = the qualified target-meeting config with the largest occupied context reached in its passing stability runs.
GLM targetbest lab C1 decode of a 1-card config against the registry speed gate (pins.json). 3-card configs are shown separately and do not satisfy the 1x RTX 3090 request.
Agentic-usablequalified and a passing stability run reached >= 128k occupied tokens; otherwise the label is 'configured for N k'.
Kit bandtop-1 >= 0.987 and mean KL <= 0.0013 (Flash-Next, quality_panel records).
GLM quality floorsolve-rate ratio >= 80% of the reference (metric solve_rate_ratio on quality_eval records, else computed from the model's eligibility=reference quality_eval).
Dropped configa config with a failed or crashed record and no record carrying a positive eligibility.

Pins

0xSero/local-ai-registryd21258dd744e7c78be28177c90060c6af8e10b7e
0xSero/local-ai-recipe-kitef883d269e50ecbea290f919f095e2f3ca633b42
SWE-agent/mini-swe-agent{"commit": "04d809ceab9df28f9adaed044884180159172930", "nearest_tag": "v2.4.6"}
panel qwen3.8-flash-next-exl3{"path": "reference/qwen3.8-flash-next-exl3-ref-panel.json", "sha256": "3350eef4e980779256a0c1e872d9345d0d21bdf43690e1958526aef17e02488d"}
panel glm-5.3-flash-exl3{"source": "0xSero/local-ai-recipe-kit PR #3 @ b5d0d3c2d567c4d4c1a295acc0a0fc86470b9097", "path": "artifacts/glm-5.3-flash-exl3-ref-panel.json", "sha256": "3445baefa35851f7f431b4f46fc2266a9d234d79526b1d4fc1216093f552f269"}
lab gatesload, chat, reasoning, tools, context, speed
lab speed_min_tps15.0
lab speedC1 streamed decode over first 30 s after first token, temperature 0.8, uncapped answer
lab contextneedle in a prompt of ~0.85*ctx; pass if recalled and prompt_tokens >= 0.6*ctx
lab chatfinish_reason=stop with non-empty content
lab reasoningseparate reasoning_content and 17*23=391 in content
lab toolsget_weather(city=Paris) call, then tool result (17) used in reply
lab prefill_definitioncontext-gate prompt_tokens / total request seconds (includes generation)
kit early_exit_threshold0.95
kit early_exit_policydisabled during exploration (spec §4)

pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78, pinned 2026-10-02

Protocol (from SPEC.md)

2. Hardware ("the ryzen", dedicated to this work)

PartDetail
CPURyzen 9 5950X, 16C/32T, AVX2, no AVX-512
BoardASUS Pro WS X570-ACE
RAM128 GB DDR4-3200, dual-channel (4×32), ~125 GiB MemTotal
GPU3× RTX 3090 24 GB, driver 610.57.04
PCIe todayGPU0 04:00.0 x8 behind the X570 chipset; GPU1 0b:00.0 x8 CPU (drives display); GPU2 0c:00.0 x8 CPU
PCIe x16 layoutone card alone in the primary CPU slot (operator moves cards)
Storage1 TB NVMe, ~748 GB free: active models. 20 TB ZFS HDD: unused

Identify GPUs by UUID in every result. Never benchmark on the display GPU. 3-card runs are headless: the operator logs out of Hyprland and everything is driven over SSH; each record notes display: off.

4. Definitions and metrics

  • Official speed: the numbers from lab/lab.py try --on endpoint in the registry. Decode = C1 over the first 30 s at T=0.8. Lab prefill = context-gate prompt tokens ÷ total request seconds.
  • Lab prefill is a conservative number. A value > 1000 counts as proof that prefill clears 1000. Record time-to-first-token prefill separately, with the prefix cache disabled and fresh prompts.
  • Pinned harness: at Phase 0 start, record the commit SHAs of the registry, kit, mini-swe-agent and the reference panels. Every run cites them, and the six lab gates and thresholds are those at that commit.
  • Decode floor measurement: generation-only tok/s (output tokens ÷ time spent streaming them, excluding prefill, tool execution and idle time) over rolling 60 s windows, ignoring the first 60 s. "Holds the floor" = every window ≥ the floor. This is separate from official lab C1 and from the whole-run average.
  • Supplementary speed: the kit PROTOCOL.md sweep (prefill 8k/16k/32k/64k, decode C1–C4), always run with early exit disabled while exploring.
  • Real-world speed: decode measured across the whole agentic run.
  • Context is reported as occupied tokens (the prompt actually sent), alongside the configured limit. Maximum tested: 260k. The label "agentic-usable" is awarded only after a config passes §5 qualification at ≥ 128k occupied tokens; otherwise it shows "configured for N k". Shorter profiles are still published.
  • Occupied context = prompt tokens counted by the served model's own tokenizer (usage.prompt_tokens from the server), excluding generated tokens. A max-context claim needs evidence that a request actually reached that occupancy, with ≥ 4k tokens of generation headroom left.
  • MTP / speculative decoding: always report accepted tokens/s, acceptance rate and the draft settings.

5. Stability and quality protocol

Run types
  • Stability runs: capped at 10 min each (screening and qualification below).
  • Quality evaluation (GLM solve-rate, §5 GLM quality floor): a separate run with no 10-min cap but a fixed token budget per task. Finalists only.
Agentic harness
  • mini-swe-agent, pinned commit.
  • Fixed set of 5–10 SWE-bench-Lite tasks, at least one growing past 64k context.
  • Sampling is fixed: temperature, top-p, seed where supported, max tokens.
Screening run
  • One run of ≤ 10 min.
Finalist qualification
  • 3 runs of ≤ 10 min each: one starting cold, one at the claimed maximum occupied context, one repeat.
  • Cold = container restarted (prefix/KV and expert caches empty), the n-gram row cache empty, and the OS page cache dropped (sync; echo 3 > /proc/sys/vm/drop_caches). Each record lists which caches were reset.
  • Each run works through the task set in fixed order, looping, until the 10 min wall clock runs out. An unfinished task counts as unsolved.
Pass criteria (every run)
  • No crash and no OOM.
  • 0 tool-call parse failures. The ≥ 100 tool-call minimum is counted across all runs of a config (screening + qualification + extra 10-min runs as needed), not per run.
  • No repetition loops. Detector (frozen per decision 0003): any 64-token sequence repeated ≥ 3 times within one field of a response (reasoning, content, or one tool call's arguments, each checked independently; cross-field matches are logged as diagnostics only), or the same tool call (name + arguments) issued ≥ 4 times in a row within one agent conversation (the streak resets only when a fresh conversation starts). When it triggers, save the offending response. Tokens come from the served model's tokenizer when the server exposes one, else tiktoken cl100k (recorded).
  • Error categories are reported separately: malformed tool calls (invalid argument JSON, unparsed tool-call markup, unknown tool, missing command) are tool-call parse failures; responses cut off at max_tokens without a tool call are output truncations; other agent-protocol errors are agent format errors. Only the first category is the "0 parse failures" criterion; the other two are published next to it.
  • Decode stays at or above the absolute floor for the whole run: Flash-Next ≥ 50 tok/s; GLM ≥ 15 tok/s when the config is a registry candidate.
  • All six registry lab gates still pass after the run.
GLM quality floor
  • Passes the registry tools gate.
  • Meets the parse-failure rule above.
  • Solves ≥ 80 % as many tasks as the reference on the same task set, measured in the uncapped quality evaluation with identical task IDs, sampling settings and per-task token budget for candidate and reference. If the reference solves 0 tasks, the solve-rate criterion is reported as not measurable and the config cannot be a registry candidate on quality grounds.
  • Reference: GLM exact mode (bit-exact EXL3 3.05 bpw). If exact mode cannot load, the reference is the highest-bit GGUF that fits.
  • Also report KLD and top-1 agreement where they are available.
Flash-Next quality
  • Kit band: top-1 ≥ 0.987 and KL ≤ 0.0013 against the exllamav3 reference panel (tools/score_ref_panel.py, the kit's pinned panel).
  • The band is required for the headline result and for the registry PR. Configs outside it are still submitted to the kit and published, labelled "outside band" (precedent: kit PR #1).