Current answer · as of 2026-10-12
For coding agents on one RTX 3090, run Qwen3.8-Flash-Next on the Strata engine. Tested on one active RTX 3090 at PCIe 4.0 x8 in a 3-card AM4 host (Ryzen 9 5950X, 128 GB DDR4). Pick IQ3_S for task success (48 of 50 fresh issues solved, against 39 for Qwen3.8-27B on the registry's own 208k recipe; slowest minute of agentic decode 87 tok/s) or Q2_0 for speed (slowest minute 123 tok/s; 46 of 50 solved). Both recipes are submitted to the local-ai registry with passing gates. Avoid the OrcaRouter Uncensored fine-tune for coding (14 of 50). GLM-5.3-Flash stays under 15 tok/s on every route we tried on this box.
Recipe: Flash-Next · Strata · IQ3_S (best task success) Recipe: Flash-Next · Strata · Q2_0 (fastest)
Agentic speed
123 tok/s
Slowest 60-second window of decode during a 10-minute coding-agent run.
Qwen3.8-Flash-Next · Strata · GGUF Q2_0 · 1 active RTX 3090, PCIe 4.0 x8 · 131k context
Limit: Single run per quant; Strata serves one request at a time. The previous best on this card (trellis/SGLang) was 40.9.
Evidence → Fresh-issue coding
48 / 50 vs 39 / 50
Qwen3.8-Flash-Next (Strata IQ3_S) vs Qwen3.8-27B on the registry's 208k RTX 3090 recipe, on 50 private bug fixes opened after both models were released. Predeclared comparison (27B on our 131k launch): 48 vs 36, p = 0.0018.
One pass per task, same agent, sampling and 60k-token budget; 208k comparison p = 0.012 (exploratory)
430 attempts → 50 tasks · paired 13–1 · exact McNemar p = 0.0018 · commitment 5371c3b8d1ca…
Limit: On this 50-task set only; tasks skew easy; curated by Claude Opus.
Evidence → GLM-5.3-Flash feasibility
< 15 tok/s
Best single-card decode across every tested route on this host: GGUF engines, exllamav3 exact mode, and Sero's NVMe-tier recipe (11.3 tok/s at a 90 GiB RAM cap).
GLM-5.3-Flash (321B total, 18B active) · 1 active RTX 3090 · 128 GB DDR4 · one NVMe at PCIe 4.0 x2
Limit: Hardware-bound here: NVMe first, then CPU and memory bandwidth (probe).
Evidence → How we got here 10 / 03
Fix Found and isolated a faulty CPU core Machine-check errors traced to core 2; it is taken offline at boot and CPU boost is disabled, leaving 15 cores at 3.4 GHz for every run that follows.
decision 0004 decision 0010 Finding →
10 / 04
Result Card order matters: +43–51 % prefill Putting the card on the CPU's PCIe lanes (GPU2, 13.5 GB/s) first instead of the chipset-attached card (6 GB/s) raised prefill by 43–51 % on multi-card runs.
decision 0006 Finding →
10 / 04
Negative result Flash-Next on trellis/SGLang: 55 tok/s in the lab, ~40 under a coding agent Sero's PR #144 launch reaches 55–57 tok/s lab decode and ~2,400 tok/s prefill, but the slowest minute of an agentic run stays near 40 tok/s on a PCIe x8 card. No tested x8 setting reached the 50 tok/s floor.
decision 0011 Finding →
10 / 05
Negative result GLM-5.3-Flash: under 15 tok/s on every route GGUF quants on llama.cpp and ik_llama (1–3 cards) and exllamav3 exact mode all stayed below the 15 tok/s gate.
decision 0012 Finding →
10 / 05
Fix PR #144 image ran out of VRAM in 2 of 4 runs; mitigated The image runs a 24 GB card down to 2 MiB free. Chunked prefill 8192 left 1,060 MiB free and ran 3 of 3 full sequences without a crash, decode unchanged, prefill −8 to −12 %.
decision 0013 Finding →
10 / 06
Result Public coding benchmark turned out to be memorised Flash-Next solved 30 of 30 SWE-bench-Lite tasks against 18 for Qwen3.8-27B, but its patches reproduce the original fixes almost word for word (median 100 % of the fix's added lines): evidence consistent with memorisation, not skill.
decision 0014 Finding →
10 / 08
Contribution Reported upstream to Sero Posted the PR #144 rerun with the out-of-memory finding and its mitigation, and opened a recipe-kit submission with this box's x8 data point.
decision 0015 Finding →
10 / 08
Result Strata engine: 123 and 87 tok/s at the slowest minute The Strata C++ engine with ISTA-DASLab GGUF quants held 123 tok/s (Q2_0) and 87 tok/s (IQ3_S) through 10-minute agentic runs on the same x8 card, with lab prefill 2,174–2,664 tok/s.
ticket T41 Finding →
10 / 08
Result moetier: what limits GLM here Sero's NVMe-tier GLM recipe gave 6.1 tok/s at a 55 GiB RAM cap and 11.3 at 90 GiB. Probe sampling shows the NVMe drive saturated at 55 GiB and the CPU expert lane the limit at 90 GiB.
ticket T42 Finding →
10 / 10
Tool Built a private fresh-issues benchmark 430 attempts on Python issues opened after 1 September 2026 gave 50 validated test tasks. Flash-Next on Strata IQ3_S solved 48, Qwen3.8-27B 36 (13 vs 1 discordant tasks, exact McNemar p = 0.0018, declared in advance).
decision 0016 ticket T43 Finding →
10 / 11
Negative result OrcaRouter's uncensored Flash-Next: same speed, 14 of 50 solved Run on Strata through its manual compatibility pack, the uncensored IQ3_XXS fine-tune matched IQ3_S on speed (86 tok/s floor) but solved 14 of our 50 fresh tasks, with 72 truncated replies and 14 malformed tool calls.
ticket T46 Finding →
10 / 12
Result 27B on the registry's own recipe: 39 of 50 Re-running Qwen3.8-27B with the registry's exact 208k RTX 3090 recipe gave 39 of 50 (our 131k launch: 36). Strata IQ3_S still leads 48 to 39 (10 tasks vs 1, p = 0.012).
ticket T48 Finding →
10 / 12
Contribution Strata recipes and moetier data sent upstream Opened a registry PR with Strata IQ3_S and Q2_0 recipes for the RTX 3090 (six gates pass, 75 and 99.7 tok/s), and a moetier PR with this box's hardware record, the two GLM NVMe runs and a simulator calibration point.
ticket T47 ticket T42 Finding →
What more hardware would test No cost: Move the model NVMe to a CPU-attached x4 slot. Simulated 10.8 tok/s instead of 7.0 at a 55 GiB cap; little effect at 90 GiB, where the CPU binds. Medium: Workstation platform: PCIe 4.0 x16 per GPU, 8-channel memory (e.g. EPYC). trellis Flash-Next towards Sero's x16 measurement on the same image (kit C1 61 tok/s, lab 67.5); GLM 23–27 tok/s (simulated on his EPYC box). Large: One 96 GB Blackwell card (RTX PRO 6000). GLM 36–40 tok/s simulated (a lower bound; the 3090's fixed per-token time was used). Flash-Next experts mostly resident. Not recommended: A 4th RTX 3090. No gain for these engines. Constraints and success criteria → Next A second fresh slice (October issues) and the first non-Python slice (TypeScript, Go, Rust). Qwen 4: download, launch and screen on day one with the same suite. Repeat the Strata speed runs (3 runs incl. a cold start) to qualify them under the site's stability rule. Run the OrcaRouter GGUF on llama.cpp to tell whether the fine-tune or Strata's compatibility repack causes its failures. Demos for Sero: a recorded Strata coding-agent run and a long-context codebase Q&A. 281 result records; latest run 2026-10-12 11:48 AEST. All data →
Findings
Speed Coding quality GLM-5.3-Flash Hardware Which setup runs coding agents fastest? Measured on one RTX 3090 at PCIe 4.0 x8 (GPU2). The number that matters for an agent is the floor: the slowest 60-second window of decode during a 10-minute agentic coding run, not the lab's first 30 seconds.
Flash-Next is a 180B mixture-of-experts model; only ~11 of 512 experts run per token, so most expert weights live in host RAM and stream to the card. On trellis/SGLang that stream saturates the x8 link: probe samples show PCIe host-to-GPU at its 14.3 GB/s ceiling at peaks while the CPU and NVMe sit idle. The agentic floor stays at ~40 tok/s (Sero's x16 box reports 61 tok/s lab decode on the same image). Strata runs ISTA-DASLab's 2–3.5-bit GGUF quants with its own expert cache and MTP speculative decoding (≈ 87 % draft acceptance in our smoke test). Its floors are 123 tok/s (Q2_0) and 87 tok/s (IQ3_S) on the same card. Qwen3.8-27B is dense and fits on the card: 111 tok/s floor with MTP, but a smaller real context (135k-token KV pool). The PR #144 image needs --chunked-prefill-size 8192 on a 24 GB card (see the fix on the timeline). Agentic floor by setup Qwen3.8-Flash-Next GLM-5.3-Flash Qwen3.8-27B
0 50 100 150 Flash-Next GGUF Q2_0 · strata strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2: slowest 60 s window 123 tok/s, 116 tool calls 123 27B EXL3 3.00 bpw · sglang-exl3 sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2: slowest 60 s window 111 tok/s, 137 tool calls 111 Flash-Next GGUF IQ3_S · strata strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2: slowest 60 s window 87.3 tok/s, 77 tool calls 87.3 Flash-Next GGUF IQ3_XXS · strata strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2: slowest 60 s window 86.0 tok/s, 50 tool calls 86.0 Flash-Next EXL3 3.05 bpw · trellis/SGLang sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2: slowest 60 s window 40.9 tok/s, 72 tool calls 40.9 Flash-Next GGUF IQ4_XS · llama.cpp · 3 cards llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: slowest 60 s window 38.6 tok/s, 70 tool calls 38.6 Flash-Next GGUF IQ4_XS · llama.cpp fn-llamacpp-iq4xs-128k-1x-fit-gpu2: slowest 60 s window 23.8 tok/s, 49 tool calls 23.8 GLM GGUF IQ2_M · ik_llama.cpp · 3 cards ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: slowest 60 s window 13.3 tok/s, 0 tool calls 13.3 GLM GGUF IQ2_M · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012: slowest 60 s window 11.4 tok/s, 26 tool calls 11.4 GLM GGUF IQ3_XXS · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012: slowest 60 s window 9.86 tok/s, 22 tool calls 9.86 GLM GGUF IQ2_M · ik_llama.cpp ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2: slowest 60 s window 9.46 tok/s, 0 tool calls 9.46 GLM GGUF IQ2_M · llama.cpp glm-llamacpp-iq2m-128k-1x-gpu2: slowest 60 s window 8.91 tok/s, 24 tool calls 8.91 GLM GGUF Q2_K · llama.cpp llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2: slowest 60 s window 8.80 tok/s, 19 tool calls 8.80 GLM GGUF IQ4_XS · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012: slowest 60 s window 7.81 tok/s, 25 tool calls 7.81 target 50 tok/s, slowest 60-second window of a 10-minute agentic coding run Table view Config Floor tok/s Run Flash-Next GGUF Q2_0 · strata 123 strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 27B EXL3 3.00 bpw · sglang-exl3 111 sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Flash-Next GGUF IQ3_S · strata 87.3 strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Flash-Next GGUF IQ3_XXS · strata 86.0 strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Flash-Next EXL3 3.05 bpw · trellis/SGLang 40.9 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Flash-Next GGUF IQ4_XS · llama.cpp · 3 cards 38.6 llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Flash-Next GGUF IQ4_XS · llama.cpp 23.8 fn-llamacpp-iq4xs-128k-1x-fit-gpu2 GLM GGUF IQ2_M · ik_llama.cpp · 3 cards 13.3 ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 GLM GGUF IQ2_M · llama.cpp · 3 cards 11.4 llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 GLM GGUF IQ3_XXS · llama.cpp · 3 cards 9.86 llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 GLM GGUF IQ2_M · ik_llama.cpp 9.46 ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 GLM GGUF IQ2_M · llama.cpp 8.91 glm-llamacpp-iq2m-128k-1x-gpu2 GLM GGUF Q2_K · llama.cpp 8.80 llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 GLM GGUF IQ4_XS · llama.cpp · 3 cards 7.81 llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 GLM GGUF IQ3_XXS · llama.cpp 7.65 llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 GLM EXL3 3.05 bpw · exllamav3 3.36 exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
RAM, CPU and VRAM per phase CPU cores busy (of 15 cores / 30 threads online) prefill decode
0 5 10 15 20 Flash-Next GGUF Q2_0 · strata · Prefill: 1.33 CPU cores busy 1.33 Flash-Next GGUF Q2_0 · strata · Decode: 15.2 CPU cores busy 15.2 Strata Q2_0 Flash-Next GGUF IQ3_S · strata · Prefill: 1.29 CPU cores busy 1.29 Flash-Next GGUF IQ3_S · strata · Decode: 15.2 CPU cores busy 15.2 Strata IQ3_S 27B EXL3 3.00 bpw · sglang-exl3 · Prefill: 1.10 CPU cores busy 1.10 27B EXL3 3.00 bpw · sglang-exl3 · Decode: 1.11 CPU cores busy 1.11 27B EXL3 Flash-Next EXL3 3.05 bpw · trellis/SGLang · Prefill: 1.28 CPU cores busy 1.28 Flash-Next EXL3 3.05 bpw · trellis/SGLang · Decode: 1.26 CPU cores busy 1.26 Flash-Next trellis Flash-Next GGUF IQ3_XXS · strata · Prefill: 1.34 CPU cores busy 1.34 Flash-Next GGUF IQ3_XXS · strata · Decode: 15.2 CPU cores busy 15.2 Strata IQ3_XXS cores busy (mean)
Memory during decode VRAM (GiB) host RAM: model and buffers (GiB)
0 20 40 60 80 Flash-Next GGUF Q2_0 · strata: VRAM 23.6 GiB 23.6 Flash-Next GGUF Q2_0 · strata: host RAM (model and buffers) 33.6 GiB 33.6 Strata Q2_0 Flash-Next GGUF IQ3_S · strata: VRAM 23.7 GiB 23.7 Flash-Next GGUF IQ3_S · strata: host RAM (model and buffers) 48.8 GiB 48.8 Strata IQ3_S 27B EXL3 3.00 bpw · sglang-exl3: VRAM 22.6 GiB 22.6 27B EXL3 3.00 bpw · sglang-exl3: host RAM (model and buffers) 5.37 GiB 5.37 27B EXL3 Flash-Next EXL3 3.05 bpw · trellis/SGLang: VRAM 22.9 GiB 22.9 Flash-Next EXL3 3.05 bpw · trellis/SGLang: host RAM (model and buffers) 57.4 GiB 57.4 Flash-Next trellis Flash-Next GGUF IQ3_XXS · strata: VRAM 23.7 GiB 23.7 Flash-Next GGUF IQ3_XXS · strata: host RAM (model and buffers) 51.8 GiB 51.8 Strata IQ3_XXS GiB
Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed Flash-Next GGUF Q2_0 · strata Idle (loaded) 23.5 GiB 33.2 GiB 39.1 GiB 0.1 (p95 0.27) 0 % 0 GB/s 100 W – Prefill 23.6 GiB 33.6 GiB 39.1 GiB 1.33 (p95 2.88) 93.4 % 8.42 GB/s 348 W 2,584 tok/s Decode 23.6 GiB 33.6 GiB 39.1 GiB 15.2 (p95 15.6) 99.6 % 0.78 GB/s 381 W 123 tok/s Flash-Next GGUF IQ3_S · strata Idle (loaded) 23.6 GiB 48.4 GiB 55.2 GiB 0.19 (p95 0.29) 0 % 0 GB/s 109 W – Prefill 23.6 GiB 48.9 GiB 55.2 GiB 1.29 (p95 2.60) 95.6 % 11.6 GB/s 322 W 1,964 tok/s Decode 23.7 GiB 48.8 GiB 55.2 GiB 15.2 (p95 15.6) 99.6 % 3.47 GB/s 359 W 93.8 tok/s 27B EXL3 3.00 bpw · sglang-exl3 Idle (loaded) 22.3 GiB 5.44 GiB 17.1 GiB 1.10 (p95 1.38) 0 % 0 GB/s 113 W – Prefill 22.6 GiB 5.37 GiB 17.1 GiB 1.10 (p95 1.27) 98.8 % 0.18 GB/s 384 W 984 tok/s Decode 22.6 GiB 5.37 GiB 17.1 GiB 1.11 (p95 1.28) 95.3 % 2.06 GB/s 388 W 108 tok/s Flash-Next EXL3 3.05 bpw · trellis/SGLang Idle (loaded) 21.1 GiB 57.2 GiB 54.8 GiB 1.20 (p95 2.10) 1.30 % 0.05 GB/s 152 W – Prefill 22.9 GiB 57.5 GiB 55.5 GiB 1.28 (p95 1.81) 74.5 % 9.19 GB/s 279 W 2,131 tok/s Decode 22.9 GiB 57.4 GiB 55.8 GiB 1.26 (p95 1.61) 99.6 % 6.38 GB/s 309 W 53.0 tok/s Flash-Next GGUF IQ3_XXS · strata Idle (loaded) 23.7 GiB 51.8 GiB 57.2 GiB 0.09 (p95 0.28) 0 % 0 GB/s 120 W – Prefill 23.7 GiB 51.9 GiB 57.2 GiB 1.34 (p95 2.58) 95.3 % 12.5 GB/s 320 W 1,911 tok/s Decode 23.7 GiB 51.8 GiB 57.2 GiB 15.2 (p95 15.6) 99.9 % 3.81 GB/s 352 W 92.8 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
Decode vs occupied context Qwen3.8-Flash-Next GLM-5.3-Flash Qwen3.8-27B lab C1 (first 30 s) stability: min 60 s window0 50 100 150 200 0 50 100 150 200 occupied context (k tokens) decode tok/s Flash-Next target 50 GLM gate 15 128k fn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 49.3 tok/s at 166,667 occupied · pass fn-trellis-3.05-0xsero-main-x8-gpu2 · stability min 60 s window 38.1 tok/s at 30,034 occupied · fail fn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · pass fn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.5 tok/s at 106,294 occupied · pass fn-llamacpp-iq4xs-128k-1x-fit-gpu2 · stability min 60 s window 23.8 tok/s at 16,819 occupied · fail fn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.1 tok/s at 106,294 occupied · pass glm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail glm-llamacpp-iq2m-128k-1x-gpu2 · stability min 60 s window 8.91 tok/s at 9,047 occupied · fail glm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.20 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · stability min 60 s window 8.80 tok/s at 6,658 occupied · fail llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.30 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 10.0 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · stability min 60 s window 9.46 tok/s at 1,874 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 · lab C1 decode 8.20 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.30 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · stability min 60 s window 7.65 tok/s at 4,875 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.40 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · stability min 60 s window 8.85 tok/s at 9,732 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 · lab C1 decode 8.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · lab C1 decode 12.3 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · stability min 60 s window 11.4 tok/s at 8,118 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.5 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · stability min 60 s window 9.86 tok/s at 5,309 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.4 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 · lab C1 decode 12.2 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 11.0 tok/s at 6,174 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · stability min 60 s window 8.85 tok/s at 6,632 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.00 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 7.81 tok/s at 6,852 occupied · fail llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.30 tok/s at 93,437 occupied · fail llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 56.9 tok/s at 106,294 occupied · pass llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · stability min 60 s window 38.6 tok/s at 39,226 occupied · fail llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 57.1 tok/s at 106,294 occupied · pass llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 · lab C1 decode 9.10 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 · lab C1 decode 11.2 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 14.0 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · stability min 60 s window 13.3 tok/s at 1,874 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 13.4 tok/s at 93,437 occupied · fail llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 · lab C1 decode 44.3 tok/s at 106,294 occupied · pass llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 · lab C1 decode 35.0 tok/s at 106,294 occupied · pass ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 · lab C1 decode 10.2 tok/s at 93,437 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · stability min 60 s window 37.9 tok/s at 36,327 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 54.5 tok/s at 166,667 occupied · pass exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.40 tok/s at 93,437 occupied · fail exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · stability min 60 s window 3.36 tok/s at 4,589 occupied · fail exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.50 tok/s at 93,437 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 55.8 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · stability min 60 s window 39.7 tok/s at 25,309 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 57.4 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · stability min 60 s window 40.1 tok/s at 29,027 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 53.7 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · stability min 60 s window 40.9 tok/s at 39,281 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 51.3 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 48.1 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · stability min 60 s window 37.4 tok/s at 34,417 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 49.7 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.9 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.5 tok/s at 25,179 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 49.5 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 49.9 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 53.5 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 38.8 tok/s at 28,209 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 56.4 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 55.0 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 40.4 tok/s at 30,539 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 45.0 tok/s at 166,667 occupied · pass sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 · stability min 60 s window 111 tok/s at 52,045 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 48.8 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.8 tok/s at 28,542 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 · lab C1 decode 144 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 123 tok/s at 38,712 occupied · pass strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · pass glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 · lab C1 decode 6.10 tok/s at 93,437 occupied · fail glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 · lab C1 decode 11.3 tok/s at 93,437 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · stability min 60 s window 39.0 tok/s at 28,256 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 56.1 tok/s at 166,667 occupied · pass strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 78.1 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 87.3 tok/s at 39,624 occupied · pass strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 75.5 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 · lab C1 decode 93.2 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 76.4 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 86.0 tok/s at 56,311 occupied · fail strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 73.6 tok/s at 106,294 occupied · pass sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 126 tok/s at 173,372 occupied · pass sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · stability min 60 s window 90.1 tok/s at 68,013 occupied · pass sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 139 tok/s at 173,372 occupied · pass Table view Model Config Measure Occupied tok/s Outcome Qwen3.8-Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 lab C1 decode 166,667 49.3 ✓ pass Qwen3.8-Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 stability min 60 s window 30,034 38.1 ! fail Qwen3.8-Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 lab C1 decode 166,667 55.3 ✓ pass Qwen3.8-Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 lab C1 decode 106,294 26.5 ✓ pass Qwen3.8-Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 stability min 60 s window 16,819 23.8 ! fail Qwen3.8-Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 lab C1 decode 106,294 26.1 ✓ pass GLM-5.3-Flash glm-llamacpp-iq2m-128k-1x-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash glm-llamacpp-iq2m-128k-1x-gpu2 stability min 60 s window 9,047 8.91 ! fail GLM-5.3-Flash glm-llamacpp-iq2m-128k-1x-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 lab C1 decode 93,437 9.20 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 stability min 60 s window 6,658 8.80 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 lab C1 decode 93,437 9.30 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 lab C1 decode 93,437 10.0 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 stability min 60 s window 1,874 9.46 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 lab C1 decode 93,437 8.20 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 lab C1 decode 93,437 8.30 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 stability min 60 s window 4,875 7.65 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 lab C1 decode 93,437 8.40 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 stability min 60 s window 9,732 8.85 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 lab C1 decode 93,437 8.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 lab C1 decode 93,437 12.3 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 stability min 60 s window 8,118 11.4 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 lab C1 decode 93,437 10.5 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 stability min 60 s window 5,309 9.86 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 lab C1 decode 93,437 10.4 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 lab C1 decode 93,437 12.2 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 11.8 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 stability min 60 s window 6,174 11.0 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 11.8 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 stability min 60 s window 6,632 8.85 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 8.00 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 stability min 60 s window 6,852 7.81 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 8.30 ! fail Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 lab C1 decode 106,294 56.9 ✓ pass Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 stability min 60 s window 39,226 38.6 ! fail Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 lab C1 decode 106,294 57.1 ✓ pass GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 lab C1 decode 93,437 9.10 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 lab C1 decode 93,437 11.2 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 lab C1 decode 93,437 14.0 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 stability min 60 s window 1,874 13.3 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 lab C1 decode 93,437 13.4 ! fail Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 lab C1 decode 106,294 44.3 ✓ pass Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 lab C1 decode 106,294 35.0 ✓ pass GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 lab C1 decode 93,437 10.2 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 lab C1 decode 166,667 55.6 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 stability min 60 s window 36,327 37.9 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 lab C1 decode 166,667 54.5 ✓ pass GLM-5.3-Flash exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 lab C1 decode 93,437 7.40 ! fail GLM-5.3-Flash exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 stability min 60 s window 4,589 3.36 ! fail GLM-5.3-Flash exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 lab C1 decode 93,437 7.50 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab C1 decode 166,667 55.8 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 stability min 60 s window 25,309 39.7 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab C1 decode 166,667 57.4 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 lab C1 decode 166,667 55.6 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 stability min 60 s window 29,027 40.1 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 lab C1 decode 166,667 54.6 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 lab C1 decode 166,667 53.7 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 stability min 60 s window 39,281 40.9 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 lab C1 decode 166,667 51.3 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 lab C1 decode 166,667 48.1 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 stability min 60 s window 34,417 37.4 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 lab C1 decode 166,667 49.7 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 54.9 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 25,179 39.5 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 49.5 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab C1 decode 166,667 49.9 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 53.5 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 28,209 38.8 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 56.4 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 55.0 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 30,539 40.4 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 45.0 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 stability min 60 s window 52,045 111 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 48.8 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 28,542 39.8 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 54.6 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 lab C1 decode 106,294 144 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 100 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 stability min 60 s window 38,712 123 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 100 ✓ pass GLM-5.3-Flash glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 lab C1 decode 93,437 6.10 ! fail GLM-5.3-Flash glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 lab C1 decode 93,437 11.3 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 lab C1 decode 166,667 55.3 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 stability min 60 s window 28,256 39.0 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 lab C1 decode 166,667 56.1 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 78.1 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 stability min 60 s window 39,624 87.3 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 75.5 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 93.2 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 76.4 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 stability min 60 s window 56,311 86.0 ! fail Qwen3.8-Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 73.6 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 lab C1 decode 173,372 126 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 stability min 60 s window 68,013 90.1 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 lab C1 decode 173,372 139 ✓ pass
Findings
Speed Coding quality GLM-5.3-Flash Hardware How often do the tested setups solve fresh issues? Public coding benchmarks are older than these models' training data. On SWE-bench-Lite, Flash-Next reproduces the original fixes almost verbatim, so we built a private set of issues opened after both models were released.
Evidence 430 attempts → 62 validated → 50 test tasks (how ) Issue and PR both created on or after 2026-09-01 Primary comparison declared 2026-10-09 05:50 AEST, before any run Paired outcomes: 35 both · 13 only Strata · Flash-Next IQ3_S · 1 only Qwen3.8-27B · EXL3 3.00 bpw · 131k launch · 1 neither Tasks private; commitment 5371c3b8d1caedc1… Curated by Claude Opus (semi-private for Anthropic models) Fifty real bug fixes from open-source Python projects, where both the GitHub issue and the pull request that fixed it were created on or after 1 September 2026. Both Qwen3.8 models were released before that (27B on 5 August, Flash-Next on 24 August), so neither could have seen these fixes in training. The tasks stay private; only scores and a commitment hash are published.
Created
09-01 → 10-06 2026
Validated from
430 attempts
Solved tasks One uncapped pass per task, same harness and sampling for every model. Whiskers: 95 % Wilson interval (±14 points at n = 50).
Strata · Flash-Next IQ3_S Strata · Flash-Next Q2_0 Qwen3.8-27B · registry 208k recipe Qwen3.8-27B · EXL3 3.00 bpw · 131k launch Strata · OrcaRouter Uncensored IQ3_XXS
0% 20% 40% 60% 80% 100% Strata · Flash-Next IQ3_S: 48/50 solved · 95 % CI 87%–99% · 83.0 tok/s 48/50 Strata IQ3_S Strata · Flash-Next Q2_0: 46/50 solved · 95 % CI 81%–97% · 123 tok/s 46/50 Strata Q2_0 Qwen3.8-27B · registry 208k recipe: 39/50 solved · 95 % CI 65%–87% · 112 tok/s 39/50 27B (208k recipe) Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 36/50 solved · 95 % CI 58%–83% · 119 tok/s 36/50 27B (131k) Strata · OrcaRouter Uncensored IQ3_XXS: 14/50 solved · 95 % CI 17%–42% · 85.2 tok/s 14/50 Orca Uncensored solved Table view Config Solved 95 % CI Decode tok/s Tool calls Parse failures Truncations Repetition hits Strata · Flash-Next IQ3_S 48/50 87%–99% 83.0 3,366 1 1 6 Strata · Flash-Next Q2_0 46/50 81%–97% 123 2,748 0 3 7 Qwen3.8-27B · registry 208k recipe 39/50 65%–87% 112 2,598 1 27 43 Qwen3.8-27B · EXL3 3.00 bpw · 131k launch 36/50 58%–83% 119 2,682 4 39 67 Strata · OrcaRouter Uncensored IQ3_XXS 14/50 17%–42% 85.2 3,084 14 72 14
Primary comparison, decided before any run Strata · Flash-Next IQ3_S vs Qwen3.8-27B · EXL3 3.00 bpw · 131k launch on the same 50 tasks. Exact McNemar test on the tasks where exactly one of them succeeded.
35 Both solved
13 Only Strata IQ3_S
1 Only 27B (131k)
1 Neither
p = 0.0018 (13 vs 1 discordant tasks; holds after Bonferroni correction for 3 comparisons)
Exploratory: Strata · Flash-Next IQ3_S vs Strata · Flash-Next Q2_0 3–1, p = 0.625; Strata · Flash-Next IQ3_S vs Qwen3.8-27B · registry 208k recipe 10–1, p = 0.012; Strata · Flash-Next IQ3_S vs Strata · OrcaRouter Uncensored IQ3_XXS 35–1, p = 0.000.
By difficulty Difficulty from the size of the upstream fix: easy = 1 file, ≤ 10 lines; medium = 1–2 files, ≤ 50 lines; hard = 3+ files or more than 50 lines. Bar labels are solved counts.
Strata · Flash-Next IQ3_S Strata · Flash-Next Q2_0 Qwen3.8-27B · registry 208k recipe Qwen3.8-27B · EXL3 3.00 bpw · 131k launch Strata · OrcaRouter Uncensored IQ3_XXS
0% 20% 40% 60% 80% 100% Strata · Flash-Next IQ3_S: 23/24 easy 23 Strata · Flash-Next Q2_0: 23/24 easy 23 Qwen3.8-27B · registry 208k recipe: 21/24 easy 21 Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 19/24 easy 19 Strata · OrcaRouter Uncensored IQ3_XXS: 7/24 easy 7 easy (24) Strata · Flash-Next IQ3_S: 19/20 medium 19 Strata · Flash-Next Q2_0: 18/20 medium 18 Qwen3.8-27B · registry 208k recipe: 14/20 medium 14 Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 14/20 medium 14 Strata · OrcaRouter Uncensored IQ3_XXS: 4/20 medium 4 medium (20) Strata · Flash-Next IQ3_S: 6/6 hard 6 Strata · Flash-Next Q2_0: 5/6 hard 5 Qwen3.8-27B · registry 208k recipe: 4/6 hard 4 Qwen3.8-27B · EXL3 3.00 bpw · 131k launch: 3/6 hard 3 Strata · OrcaRouter Uncensored IQ3_XXS: 3/6 hard 3 hard (6) share solved
Why a fresh set: memorised fixes on the public benchmark Share of the upstream fix's added lines that the model's patch reproduces word for word (median over tasks). On 30 public SWE-bench-Lite tasks Flash-Next, at every quantisation, rewrites the original fix almost verbatim. On the fresh slice it writes its own.
SWE-bench-Lite (public) fresh · Strata · Flash-Next IQ3_S fresh · Strata · Flash-Next Q2_0 fresh · Qwen3.8-27B · EXL3 3.00 bpw · 131k launch fresh · Strata · OrcaRouter Uncensored IQ3_XXS
0% 20% 40% 60% 80% 100% SWE-bench-Lite (public, 2023): median 100% of the upstream fix reproduced verbatim 100% Fresh slice: median 20% reproduced verbatim 20% Strata IQ3_S SWE-bench-Lite (public, 2023): median 100% of the upstream fix reproduced verbatim 100% Fresh slice: median 17% reproduced verbatim 17% Strata Q2_0 SWE-bench-Lite (public, 2023): median 14% of the upstream fix reproduced verbatim 14% Fresh slice: median 17% reproduced verbatim 17% 27B (131k) SWE-bench-Lite (public, 2023): median 100% of the upstream fix reproduced verbatim 100% Fresh slice: median 10% reproduced verbatim 10% Orca Uncensored median verbatim share of the upstream fix
Table view Config Lite overlap Lite solved Fresh overlap Fresh solved Strata · Flash-Next IQ3_S 100% (18/30 ≥ 90 %) 29/30 20% (3/50 ≥ 90 %) 48/50 Strata · Flash-Next Q2_0 100% (18/30 ≥ 90 %) 28/30 17% (2/50 ≥ 90 %) 46/50 Qwen3.8-27B · EXL3 3.00 bpw · 131k launch 14% (4/27 ≥ 90 %) 18/30 17% (4/50 ≥ 90 %) 36/50 Strata · OrcaRouter Uncensored IQ3_XXS 100% (17/21 ≥ 90 %) 21/30 10% (3/50 ≥ 90 %) 14/50
Not completed: Flash-Next · trellis EXL3 3.05 bpw: stopped by the operator at 9 of 50 tasks (same weights as Strata, ~43 tok/s; not expected to differ measurably at n = 50).
Older evidence: public SWE-bench-Lite comparison (memorised; superseded).
Findings
Speed Coding quality GLM-5.3-Flash Hardware Is GLM-5.3-Flash practical on this machine? No: the best route on one RTX 3090 here is 11.3 tok/s decode. GLM-5.3-Flash has 321B parameters (18B active, 288 experts per layer), so even at ~3 bits most of it lives in RAM or on disk.
GGUF on llama.cpp / ik_llama (IQ2_M … IQ4_XS, 1–3 cards): 7.7–13.3 tok/s at the slowest minute. exllamav3 exact mode (all experts in RAM): 7.4 tok/s lab decode. Sero's NVMe-tier recipe (moetier, glm53-flash-offload v4.1): 6.1 tok/s at his 55 GiB RAM budget — our single NVMe (PCIe 4.0 x2 behind the chipset) is saturated — and 11.3 tok/s at 90 GiB, where the CPU expert lane becomes the limit. Quality is exact (reference panel top-1 1.0). The moetier simulator predicted 7.0 and 15.0; it is calibrated on an 8-channel EPYC, so it overestimates our 2-channel, 13-thread CPU lane. GLM-5.3-Flash: best lab decode per route 0 5 10 15 20 GGUF IQ2_M · ik_llama.cpp · 3 cards ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: lab C1 decode 14.0 tok/s 14.0 GGUF IQ2_M · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012: lab C1 decode 12.3 tok/s 12.3 EXL3 3.05 bpw · NVMe tier 90 GiB glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2: lab C1 decode 11.3 tok/s 11.3 GGUF IQ3_XXS · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012: lab C1 decode 10.5 tok/s 10.5 GGUF IQ2_M · ik_llama.cpp ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2: lab C1 decode 10.0 tok/s 10.0 GGUF IQ2_M · llama.cpp glm-llamacpp-iq2m-128k-1x-gpu2: lab C1 decode 9.60 tok/s 9.60 GGUF Q2_K · llama.cpp llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2: lab C1 decode 9.30 tok/s 9.30 GGUF IQ3_XXS · llama.cpp llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2: lab C1 decode 8.40 tok/s 8.40 GGUF IQ4_XS · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012: lab C1 decode 8.30 tok/s 8.30 EXL3 3.05 bpw · exllamav3 exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2: lab C1 decode 7.50 tok/s 7.50 EXL3 3.05 bpw · NVMe tier 55 GiB glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2: lab C1 decode 6.10 tok/s 6.10 gate 15 lab C1 decode, tok/s What limits each setup (probe) GPU busy PCIe host→GPU, % of measured 13.5 GB/s NVMe read, % of 3.36 GB/s CPU busy
0% 20% 40% 60% 80% 100% GLM-5.3 · NVMe recipe · 55 GiB RAM: GPU busy 87 % of samples with a kernel running 87% GLM-5.3 · NVMe recipe · 55 GiB RAM: PCIe in 3.34 GB/s of 13.5 (p95 6.17) 25% GLM-5.3 · NVMe recipe · 55 GiB RAM: NVMe read 3.08 GB/s of 3.36 92% GLM-5.3 · NVMe recipe · 55 GiB RAM: CPU 62% busy (expert-compute cores) 62% GLM-5.3 55 GiB GLM-5.3 · NVMe recipe · 90 GiB RAM: GPU busy 96 % of samples with a kernel running 96% GLM-5.3 · NVMe recipe · 90 GiB RAM: PCIe in 6.42 GB/s of 13.5 (p95 14.27) 48% GLM-5.3 · NVMe recipe · 90 GiB RAM: NVMe read 1.31 GB/s of 3.36 39% GLM-5.3 · NVMe recipe · 90 GiB RAM: CPU 83% busy (expert-compute cores) 83% GLM-5.3 90 GiB Flash-Next · trellis · agentic run: GPU busy 89 % of samples with a kernel running 89% Flash-Next · trellis · agentic run: PCIe in 7.77 GB/s of 13.5 (p95 14.25) 58% Flash-Next · trellis · agentic run: NVMe read 0.0 GB/s of 3.36 0% Flash-Next · trellis · agentic run: CPU 4% busy (all cores) 4% Flash-Next agentic share of ceiling
Table view Arm Phase GPU PCIe NVMe CPU GLM-5.3 · NVMe recipe · 55 GiB RAM kit 87 % of samples with a kernel running 3.34 GB/s of 13.5 (p95 6.17) 3.08 GB/s of 3.36 62% busy (expert-compute cores) GLM-5.3 · NVMe recipe · 90 GiB RAM kit 96 % of samples with a kernel running 6.42 GB/s of 13.5 (p95 14.27) 1.31 GB/s of 3.36 83% busy (expert-compute cores) Flash-Next · trellis · agentic run stab 89 % of samples with a kernel running 7.77 GB/s of 13.5 (p95 14.25) 0.0 GB/s of 3.36 4% busy (all cores)
Findings
Speed Coding quality GLM-5.3-Flash Hardware Which hardware limits affect performance? The same card behaves differently depending on where it sits and what feeds it.
PCIe lanes: GPU2 (CPU lanes, x8) measures 13.5 GB/s host-to-device; GPU0 behind the X570 chipset 6.0 GB/s. Expert-streaming engines are bound by this link (trellis Flash-Next). Card order: using the CPU-attached card first gave +43–51 % prefill on multi-card runs. CPU: a faulty core (machine-check errors) is offline and boost is off for stability, so CPU-lane engines run on 15 cores at 3.4 GHz. NVMe: the 1 TB KC3000 links at PCIe 4.0 x2 behind the chipset (≈ 3.4 GB/s); it is the bottleneck for NVMe-tier GLM at 55 GiB. A 4th RTX 3090 would not help the single-GPU engines tested here: none pools VRAM across cards. RAM, CPU and VRAM per phase CPU cores busy (of 15 cores / 30 threads online) prefill decode
0 5 10 15 20 Flash-Next GGUF Q2_0 · strata · Prefill: 1.33 CPU cores busy 1.33 Flash-Next GGUF Q2_0 · strata · Decode: 15.2 CPU cores busy 15.2 Strata Q2_0 Flash-Next GGUF IQ3_S · strata · Prefill: 1.29 CPU cores busy 1.29 Flash-Next GGUF IQ3_S · strata · Decode: 15.2 CPU cores busy 15.2 Strata IQ3_S 27B EXL3 3.00 bpw · sglang-exl3 · Prefill: 1.10 CPU cores busy 1.10 27B EXL3 3.00 bpw · sglang-exl3 · Decode: 1.11 CPU cores busy 1.11 27B EXL3 Flash-Next EXL3 3.05 bpw · trellis/SGLang · Prefill: 1.28 CPU cores busy 1.28 Flash-Next EXL3 3.05 bpw · trellis/SGLang · Decode: 1.26 CPU cores busy 1.26 Flash-Next trellis Flash-Next GGUF IQ3_XXS · strata · Prefill: 1.34 CPU cores busy 1.34 Flash-Next GGUF IQ3_XXS · strata · Decode: 15.2 CPU cores busy 15.2 Strata IQ3_XXS cores busy (mean)
Memory during decode VRAM (GiB) host RAM: model and buffers (GiB)
0 20 40 60 80 Flash-Next GGUF Q2_0 · strata: VRAM 23.6 GiB 23.6 Flash-Next GGUF Q2_0 · strata: host RAM (model and buffers) 33.6 GiB 33.6 Strata Q2_0 Flash-Next GGUF IQ3_S · strata: VRAM 23.7 GiB 23.7 Flash-Next GGUF IQ3_S · strata: host RAM (model and buffers) 48.8 GiB 48.8 Strata IQ3_S 27B EXL3 3.00 bpw · sglang-exl3: VRAM 22.6 GiB 22.6 27B EXL3 3.00 bpw · sglang-exl3: host RAM (model and buffers) 5.37 GiB 5.37 27B EXL3 Flash-Next EXL3 3.05 bpw · trellis/SGLang: VRAM 22.9 GiB 22.9 Flash-Next EXL3 3.05 bpw · trellis/SGLang: host RAM (model and buffers) 57.4 GiB 57.4 Flash-Next trellis Flash-Next GGUF IQ3_XXS · strata: VRAM 23.7 GiB 23.7 Flash-Next GGUF IQ3_XXS · strata: host RAM (model and buffers) 51.8 GiB 51.8 Strata IQ3_XXS GiB
Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed Flash-Next GGUF Q2_0 · strata Idle (loaded) 23.5 GiB 33.2 GiB 39.1 GiB 0.1 (p95 0.27) 0 % 0 GB/s 100 W – Prefill 23.6 GiB 33.6 GiB 39.1 GiB 1.33 (p95 2.88) 93.4 % 8.42 GB/s 348 W 2,584 tok/s Decode 23.6 GiB 33.6 GiB 39.1 GiB 15.2 (p95 15.6) 99.6 % 0.78 GB/s 381 W 123 tok/s Flash-Next GGUF IQ3_S · strata Idle (loaded) 23.6 GiB 48.4 GiB 55.2 GiB 0.19 (p95 0.29) 0 % 0 GB/s 109 W – Prefill 23.6 GiB 48.9 GiB 55.2 GiB 1.29 (p95 2.60) 95.6 % 11.6 GB/s 322 W 1,964 tok/s Decode 23.7 GiB 48.8 GiB 55.2 GiB 15.2 (p95 15.6) 99.6 % 3.47 GB/s 359 W 93.8 tok/s 27B EXL3 3.00 bpw · sglang-exl3 Idle (loaded) 22.3 GiB 5.44 GiB 17.1 GiB 1.10 (p95 1.38) 0 % 0 GB/s 113 W – Prefill 22.6 GiB 5.37 GiB 17.1 GiB 1.10 (p95 1.27) 98.8 % 0.18 GB/s 384 W 984 tok/s Decode 22.6 GiB 5.37 GiB 17.1 GiB 1.11 (p95 1.28) 95.3 % 2.06 GB/s 388 W 108 tok/s Flash-Next EXL3 3.05 bpw · trellis/SGLang Idle (loaded) 21.1 GiB 57.2 GiB 54.8 GiB 1.20 (p95 2.10) 1.30 % 0.05 GB/s 152 W – Prefill 22.9 GiB 57.5 GiB 55.5 GiB 1.28 (p95 1.81) 74.5 % 9.19 GB/s 279 W 2,131 tok/s Decode 22.9 GiB 57.4 GiB 55.8 GiB 1.26 (p95 1.61) 99.6 % 6.38 GB/s 309 W 53.0 tok/s Flash-Next GGUF IQ3_XXS · strata Idle (loaded) 23.7 GiB 51.8 GiB 57.2 GiB 0.09 (p95 0.28) 0 % 0 GB/s 120 W – Prefill 23.7 GiB 51.9 GiB 57.2 GiB 1.34 (p95 2.58) 95.3 % 12.5 GB/s 320 W 1,911 tok/s Decode 23.7 GiB 51.8 GiB 57.2 GiB 15.2 (p95 15.6) 99.9 % 3.81 GB/s 352 W 92.8 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
What limits each setup (probe) GPU busy PCIe host→GPU, % of measured 13.5 GB/s NVMe read, % of 3.36 GB/s CPU busy
0% 20% 40% 60% 80% 100% GLM-5.3 · NVMe recipe · 55 GiB RAM: GPU busy 87 % of samples with a kernel running 87% GLM-5.3 · NVMe recipe · 55 GiB RAM: PCIe in 3.34 GB/s of 13.5 (p95 6.17) 25% GLM-5.3 · NVMe recipe · 55 GiB RAM: NVMe read 3.08 GB/s of 3.36 92% GLM-5.3 · NVMe recipe · 55 GiB RAM: CPU 62% busy (expert-compute cores) 62% GLM-5.3 55 GiB GLM-5.3 · NVMe recipe · 90 GiB RAM: GPU busy 96 % of samples with a kernel running 96% GLM-5.3 · NVMe recipe · 90 GiB RAM: PCIe in 6.42 GB/s of 13.5 (p95 14.27) 48% GLM-5.3 · NVMe recipe · 90 GiB RAM: NVMe read 1.31 GB/s of 3.36 39% GLM-5.3 · NVMe recipe · 90 GiB RAM: CPU 83% busy (expert-compute cores) 83% GLM-5.3 90 GiB Flash-Next · trellis · agentic run: GPU busy 89 % of samples with a kernel running 89% Flash-Next · trellis · agentic run: PCIe in 7.77 GB/s of 13.5 (p95 14.25) 58% Flash-Next · trellis · agentic run: NVMe read 0.0 GB/s of 3.36 0% Flash-Next · trellis · agentic run: CPU 4% busy (all cores) 4% Flash-Next agentic share of ceiling
Table view Arm Phase GPU PCIe NVMe CPU GLM-5.3 · NVMe recipe · 55 GiB RAM kit 87 % of samples with a kernel running 3.34 GB/s of 13.5 (p95 6.17) 3.08 GB/s of 3.36 62% busy (expert-compute cores) GLM-5.3 · NVMe recipe · 90 GiB RAM kit 96 % of samples with a kernel running 6.42 GB/s of 13.5 (p95 14.27) 1.31 GB/s of 3.36 83% busy (expert-compute cores) Flash-Next · trellis · agentic run stab 89 % of samples with a kernel running 7.77 GB/s of 13.5 (p95 14.25) 0.0 GB/s of 3.36 4% busy (all cores)
Measured host → device bandwidth per card 0 5 10 15 GPU0 · bus 00000000:04:00.0 · x8 20261002T205350Z-hw-bw-gpu0-current-hw_bw: 6.03 GB/s host to device 6.03 GPU1 · bus 00000000:0B:00.0 · x8 20261004T091142Z-hw-bw-gpu1-current-hw_bw: 13.47 GB/s host to device 13.5 GPU2 · bus 00000000:0C:00.0 · x8 20261002T205342Z-hw-bw-gpu2-current-hw_bw: 13.47 GB/s host to device 13.5 GB/s host → device Recipes Tested launch commands with the numbers they produced on this box. Commands are the exact ones the harness ran (GPU UUID and paths are this box's; adjust them). Measured on one RTX 3090 at PCIe 4.0 x8 unless noted.
Flash-Next · Strata · IQ3_S (best task success) Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json
Resources during execution Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed Flash-Next GGUF IQ3_S · strata Idle (loaded) 23.6 GiB 48.4 GiB 55.2 GiB 0.19 (p95 0.29) 0 % 0 GB/s 109 W – Prefill 23.6 GiB 48.9 GiB 55.2 GiB 1.29 (p95 2.60) 95.6 % 11.6 GB/s 322 W 1,964 tok/s Decode 23.7 GiB 48.8 GiB 55.2 GiB 15.2 (p95 15.6) 99.6 % 3.47 GB/s 359 W 93.8 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
Needs: 1× 24 GB NVIDIA GPU (driver ≥ 580), ~60 GB free RAM, 90 GB disk, Docker with the NVIDIA runtime.
One-time setup git clone https://github.com/Niko1221/Strata && cd Strata && git checkout d5ea7133741e67743c0e886bb426c0ce8d69cf6c
docker build -t local-ai-bench/strata:v0.1.40.3-sm86 --build-arg CUDA_ARCHITECTURES=86 --build-arg BUILD_VISION=0 .
hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF --revision ed59f92082b1e93c0e96d60a8b11aab089b52f09 --include 'IQ3_S/*' --local-dir $MODELS/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF Run recorded by tools/run.py (artifacts/runs/20261011T191447Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Learnings First start needs network: it builds a pack in /data and downloads ~5 GB of MTP draft-layer tensors from Qwen/Qwen3.8-Flash-Next (not revision-pinned). Later starts are offline. Needs --ulimit memlock=-1 (pins 34–50 GB of experts). Serves one request at a time; set "parallel" in /data/config for concurrency. No logprobs on the CUDA server. The first streamed chunk is an empty role delta: measure time-to-first-token on the first chunk with content. Flash-Next · Strata · Q2_0 (fastest) Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json
Resources during execution Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed Flash-Next GGUF Q2_0 · strata Idle (loaded) 23.5 GiB 33.2 GiB 39.1 GiB 0.1 (p95 0.27) 0 % 0 GB/s 100 W – Prefill 23.6 GiB 33.6 GiB 39.1 GiB 1.33 (p95 2.88) 93.4 % 8.42 GB/s 348 W 2,584 tok/s Decode 23.6 GiB 33.6 GiB 39.1 GiB 15.2 (p95 15.6) 99.6 % 0.78 GB/s 381 W 123 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
Needs: 1× 24 GB NVIDIA GPU, ~45 GB free RAM, 72 GB disk.
One-time setup Same image as IQ3_S; download 'Q2_0/*' instead. Shard 2 is byte-identical across quants and can be hardlinked. Run recorded by tools/run.py (artifacts/runs/20261012T014808Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Learnings Same first-start and memlock notes as IQ3_S. Qwen3.8-27B · registry 208k recipe (SGLang-EXL3, MTP, vision) Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json
Needs: 1× 24 GB NVIDIA GPU, ~8 GB RAM.
Run recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Learnings Byte-faithful copy of the local-ai registry's qwen3.8-27b.sglang.208k recipe for the RTX 3090 (image c8922bd7…, 212,992 context). 39/50 on our fresh-issues benchmark; agentic floor 90 tok/s, lab decode 126–139 tok/s. Qwen3.8-27B · SGLang-EXL3 · 3.00 bpw + MTP Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json
Resources during execution Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed 27B EXL3 3.00 bpw · sglang-exl3 Idle (loaded) 22.3 GiB 5.44 GiB 17.1 GiB 1.10 (p95 1.38) 0 % 0 GB/s 113 W – Prefill 22.6 GiB 5.37 GiB 17.1 GiB 1.10 (p95 1.27) 98.8 % 0.18 GB/s 384 W 984 tok/s Decode 22.6 GiB 5.37 GiB 17.1 GiB 1.11 (p95 1.28) 95.3 % 2.06 GB/s 388 W 108 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
Needs: 1× 24 GB NVIDIA GPU, ~8 GB RAM.
Run recorded by tools/run.py (artifacts/runs/20261010T105104Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Learnings The server advertises 204,800 context but the KV pool on 24 GB holds 135,424 tokens; this launch sets 131,072 so long prompts fail clearly instead of mid-generation. At temperature 0.7 and 8,192-token replies it looped or truncated often in agentic runs (67 repetition hits on 50 tasks). Flash-Next · trellis/SGLang · EXL3 3.05 bpw (Sero's registry route) Qwen3.8-Flash-Next · launch launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json
Resources during execution Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed Flash-Next EXL3 3.05 bpw · trellis/SGLang Idle (loaded) 21.1 GiB 57.2 GiB 54.8 GiB 1.20 (p95 2.10) 1.30 % 0.05 GB/s 152 W – Prefill 22.9 GiB 57.5 GiB 55.5 GiB 1.28 (p95 1.81) 74.5 % 9.19 GB/s 279 W 2,131 tok/s Decode 22.9 GiB 57.4 GiB 55.8 GiB 1.26 (p95 1.61) 99.6 % 6.38 GB/s 309 W 53.0 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
Needs: 1× 24 GB GPU on the widest PCIe link you have, 75 GB free RAM (page-locked), 86 GB on a local NVMe.
Run recorded by tools/run.py (artifacts/runs/20261010T105600Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Learnings Use --chunked-prefill-size 8192 on 24 GB cards (the 12288 default ran out of memory in 2 of 4 runs). Keep the expert-cache reserve at 8.26 GB: Triton kernels load after the server reports ready. Only 3.05 bpw works: the offload store is 3-bit only. Load takes ~260 s; text and images supported. GLM-5.3-Flash · glm53-flash-offload v4.1 · NVMe tier, 90 GiB (experimental) GLM-5.3-Flash · launch launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json
Needs: 1× 24 GB GPU, 90 GiB RAM for the container, 125 GB weights + 117 GB record store on NVMe.
One-time setup mkdir -p $STORE && chattr +C $STORE # btrfs: no copy-on-write for O_DIRECT
docker run --rm -v $MODELS/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v $STORE:/nvx ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 pack-store Run recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Learnings 11 tok/s: usable for batch work, not interactive coding. Pin the CPU layout explicitly when cores are offline (GLM53_MAIN_CPUS, GLM53_NV_*). --memory and --memory-swap must be equal or pinned pages escape the cap. Models One card per model: what it is, which builds we ran, what we learned running it, and links to recipes and raw results.
Qwen3.8-Flash-Next
180B total: 125B language model (6B active; 512 experts, 10 routed + 1 shared per token) + 51B n-gram table + 4B MTP head
Best agentic floor here: 123 tok/s
Model card → Qwen3.8-27B
Dense 27B
Best agentic floor here: 111 tok/s
Model card → GLM-5.3-Flash
321B total, 18B active
Best agentic floor here: 13.3 tok/s
Model card → Models
Model card
Qwen3.8-Flash-Next huggingface.co/Qwen/Qwen3.8-Flash-Next · released 2026-08-24
180B total: 125B language model (6B active; 512 experts, 10 routed + 1 shared per token) + 51B n-gram table + 4B MTP head Native context 262,144 tokens Text and images Best agentic floor
123 tok/s
Best lab prefill
2,664 tok/s
Fresh-issues slice: Strata · Flash-Next IQ3_S 48/50; Strata · Flash-Next Q2_0 46/50; Strata · OrcaRouter Uncensored IQ3_XXS 14/50 details
Builds we ran Resources during execution CPU cores busy (of 15 cores / 30 threads online) prefill decode
0 5 10 15 20 Flash-Next GGUF Q2_0 · strata · Prefill: 1.33 CPU cores busy 1.33 Flash-Next GGUF Q2_0 · strata · Decode: 15.2 CPU cores busy 15.2 Strata Q2_0 Flash-Next GGUF IQ3_S · strata · Prefill: 1.29 CPU cores busy 1.29 Flash-Next GGUF IQ3_S · strata · Decode: 15.2 CPU cores busy 15.2 Strata IQ3_S Flash-Next EXL3 3.05 bpw · trellis/SGLang · Prefill: 1.28 CPU cores busy 1.28 Flash-Next EXL3 3.05 bpw · trellis/SGLang · Decode: 1.26 CPU cores busy 1.26 Flash-Next trellis Flash-Next GGUF IQ3_XXS · strata · Prefill: 1.34 CPU cores busy 1.34 Flash-Next GGUF IQ3_XXS · strata · Decode: 15.2 CPU cores busy 15.2 Strata IQ3_XXS cores busy (mean)
Memory during decode VRAM (GiB) host RAM: model and buffers (GiB)
0 20 40 60 80 Flash-Next GGUF Q2_0 · strata: VRAM 23.6 GiB 23.6 Flash-Next GGUF Q2_0 · strata: host RAM (model and buffers) 33.6 GiB 33.6 Strata Q2_0 Flash-Next GGUF IQ3_S · strata: VRAM 23.7 GiB 23.7 Flash-Next GGUF IQ3_S · strata: host RAM (model and buffers) 48.8 GiB 48.8 Strata IQ3_S Flash-Next EXL3 3.05 bpw · trellis/SGLang: VRAM 22.9 GiB 22.9 Flash-Next EXL3 3.05 bpw · trellis/SGLang: host RAM (model and buffers) 57.4 GiB 57.4 Flash-Next trellis Flash-Next GGUF IQ3_XXS · strata: VRAM 23.7 GiB 23.7 Flash-Next GGUF IQ3_XXS · strata: host RAM (model and buffers) 51.8 GiB 51.8 Strata IQ3_XXS GiB
Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed Flash-Next GGUF Q2_0 · strata Idle (loaded) 23.5 GiB 33.2 GiB 39.1 GiB 0.1 (p95 0.27) 0 % 0 GB/s 100 W – Prefill 23.6 GiB 33.6 GiB 39.1 GiB 1.33 (p95 2.88) 93.4 % 8.42 GB/s 348 W 2,584 tok/s Decode 23.6 GiB 33.6 GiB 39.1 GiB 15.2 (p95 15.6) 99.6 % 0.78 GB/s 381 W 123 tok/s Flash-Next GGUF IQ3_S · strata Idle (loaded) 23.6 GiB 48.4 GiB 55.2 GiB 0.19 (p95 0.29) 0 % 0 GB/s 109 W – Prefill 23.6 GiB 48.9 GiB 55.2 GiB 1.29 (p95 2.60) 95.6 % 11.6 GB/s 322 W 1,964 tok/s Decode 23.7 GiB 48.8 GiB 55.2 GiB 15.2 (p95 15.6) 99.6 % 3.47 GB/s 359 W 93.8 tok/s Flash-Next EXL3 3.05 bpw · trellis/SGLang Idle (loaded) 21.1 GiB 57.2 GiB 54.8 GiB 1.20 (p95 2.10) 1.30 % 0.05 GB/s 152 W – Prefill 22.9 GiB 57.5 GiB 55.5 GiB 1.28 (p95 1.81) 74.5 % 9.19 GB/s 279 W 2,131 tok/s Decode 22.9 GiB 57.4 GiB 55.8 GiB 1.26 (p95 1.61) 99.6 % 6.38 GB/s 309 W 53.0 tok/s Flash-Next GGUF IQ3_XXS · strata Idle (loaded) 23.7 GiB 51.8 GiB 57.2 GiB 0.09 (p95 0.28) 0 % 0 GB/s 120 W – Prefill 23.7 GiB 51.9 GiB 57.2 GiB 1.34 (p95 2.58) 95.3 % 12.5 GB/s 320 W 1,911 tok/s Decode 23.7 GiB 51.8 GiB 57.2 GiB 15.2 (p95 15.6) 99.9 % 3.81 GB/s 352 W 92.8 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
What we learned running it Strata is the fastest tested engine on one x8 card (123 / 87 tok/s floors) and the best tested for fresh-issue coding (48/50). ticket T41 trellis/SGLang is PCIe-bound at x8: agentic floor ~40 tok/s; wider links matter more than any setting we tuned. decision 0011 PR #144 image: use chunked prefill 8192 on 24 GB cards. decision 0013 The offload engine supports 3-bit experts only (2.05 / 4.05 bpw fail at load). decision 0011 On public SWE-bench-Lite it reproduces upstream fixes verbatim: do not use that set to judge it. decision 0014 Real context: the trellis launch held a 191k-token needle; Strata launches here use 131k. decision 0014 The OrcaRouter uncensored fine-tune runs on Strata at full speed but is unreliable for coding agents: 14/50 on our benchmark, 72 truncations, 14 malformed tool calls. ticket T46 ISTA's Coder (half the experts) is reported at 91 % of the base model's SWE-bench Verified; we stopped it after lab and TTFT as it is not expected to beat IQ3_S. ticket T45 Strata recipes for this card are in the local-ai registry PR #212 (IQ3_S 57 GB RAM, Q2_0 41 GB RAM measured at the gates). ticket T47 Recipes Flash-Next · Strata · IQ3_S (best task success) Qwen3.8-Flash-Next · launch launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json
Full recipe →
Flash-Next · trellis/SGLang · EXL3 3.05 bpw (Sero's registry route) Qwen3.8-Flash-Next · launch launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json
Full recipe →
All Qwen3.8-Flash-Next configs and runs →
Models
Model card
Qwen3.8-27B huggingface.co/Qwen/Qwen3.8-27B · released 2026-08-05
Dense 27B Fits one 24 GB card at 3.00 bpw with MTP speculative decoding Best agentic floor
111 tok/s
Best lab prefill
836 tok/s
Fresh-issues slice: Qwen3.8-27B · EXL3 3.00 bpw · 131k launch 36/50; Qwen3.8-27B · registry 208k recipe 39/50 details
Builds we ran Resources during execution CPU cores busy (of 15 cores / 30 threads online) prefill decode
0 0.25 0.5 0.75 1 1.25 27B EXL3 3.00 bpw · sglang-exl3 · Prefill: 1.10 CPU cores busy 1.10 27B EXL3 3.00 bpw · sglang-exl3 · Decode: 1.11 CPU cores busy 1.11 27B EXL3 cores busy (mean)
Memory during decode VRAM (GiB) host RAM: model and buffers (GiB)
0 5 10 15 20 25 27B EXL3 3.00 bpw · sglang-exl3: VRAM 22.6 GiB 22.6 27B EXL3 3.00 bpw · sglang-exl3: host RAM (model and buffers) 5.37 GiB 5.37 27B EXL3 GiB
Setup Phase VRAM RAM: model & buffers RAM: file cache CPU busy GPU busy PCIe in Power Speed 27B EXL3 3.00 bpw · sglang-exl3 Idle (loaded) 22.3 GiB 5.44 GiB 17.1 GiB 1.10 (p95 1.38) 0 % 0 GB/s 113 W – Prefill 22.6 GiB 5.37 GiB 17.1 GiB 1.10 (p95 1.27) 98.8 % 0.18 GB/s 384 W 984 tok/s Decode 22.6 GiB 5.37 GiB 17.1 GiB 1.11 (p95 1.28) 95.3 % 2.06 GB/s 388 W 108 tok/s
Sampled every 0.1 s (moetier probe) on GPU2, PCIe 4.0 x8. Prefill: 3 fresh ~27k-token prompts, 1 output token. Decode: 3 answers of up to 2,048 tokens. CPU busy = sum of per-thread busy fractions (spin-waiting counts as busy). RAM: container memory.stat at phase end; "file cache" is model file pages Linux can reclaim.
What we learned running it Fast and light: 121–144 tok/s decode, 111 tok/s agentic floor, 7.5 GB host RAM. decision 0014 KV pool is 135,424 tokens on 24 GB although the server advertises 204,800: launch with 131k. decision 0014 On our fresh benchmark: 39/50 with the registry's 208k recipe, 36/50 with our 131k launch (no real difference). Both loop often at temperature 0.7 (43 repetition hits on the 208k run). ticket T48 Recipes Qwen3.8-27B · registry 208k recipe (SGLang-EXL3, MTP, vision) Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json
Full recipe →
Qwen3.8-27B · SGLang-EXL3 · 3.00 bpw + MTP Qwen3.8-27B · launch launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json
Full recipe →
All Qwen3.8-27B configs and runs →
Models
Model card
GLM-5.3-Flash huggingface.co/zai-org/GLM-5.3-Flash
321B total, 18B active 45 layers (42 MoE, 288 experts, 8 active), hybrid KDA + DSA/MLA attention, 1 MTP layer Native FP8 Best lab decode
14.0 tok/s
Best agentic floor
13.3 tok/s
Best lab prefill
739 tok/s
Builds we ran What we learned running it Under 15 tok/s on every route on this host; best 13.3 tok/s floor (ik_llama IQ2_M, 3 cards). decision 0012 NVMe-tier recipe: RAM is the first lever (6.1 → 11.3 tok/s from 55 to 90 GiB), then CPU and memory bandwidth. ticket T42 Exact quality: the reference panel matches token for token (top-1 1.0). ticket T42 Recipes GLM-5.3-Flash · glm53-flash-offload v4.1 · NVMe tier, 90 GiB (experimental) GLM-5.3-Flash · launch launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json
Full recipe →
All GLM-5.3-Flash configs and runs →
Method & Hardware How every number on this site is produced. Each measurement writes a result record (pass or fail) with the exact command, image digest, weights revision and hardware state.
Speed measurements Lab gates (Sero's registry lab.py): load, chat, reasoning, tool calls, a long-context gate at ~85 % of the configured context, and decode speed over the first 30 s of a streamed answer (C1).Kit sweep (recipe-kit protocol): prefill at 8k–64k with fresh random prompts; decode at 1–4 concurrent streams and at 32k context.TTFT : time to the first token with content at 8k and 32k.Agentic floor : a 10-minute mini-swe-agent coding run; the slowest 60-second window of decode after the first minute.Probe (moetier): GPU, PCIe, NVMe and per-core CPU utilisation every 0.1 s, compared with this box's measured ceilings.Quality measurements Reference panel : teacher-forced token distributions against an exllamav3 reference (top-1 agreement, KL); engines without logprobs (Strata) cannot be scored this way.Fresh-issues benchmark : private real-world bug fixes; construction below.Public SWE-bench-Lite (30 tasks) : kept for reference only; Flash-Next has memorised it.Fresh-issues benchmark: construction How the tasks were made Harvest. A GitHub crawler listed merged pull requests in popular Python repositories and kept only those that close an issue in the same repository, where the issue and the PR were both opened on or after 1 September 2026, the PR targets the default branch, and it changes both source code and tests. Bots, backports, reverts, duplicates, docs-only changes, very large fixes and issues that already contain the fix were removed. At most three tasks per repository.Clarity review. Each issue was graded by Claude Opus on the SWE-bench Verified rubric: is the issue clear enough to fix, do the hidden tests check only what the issue asks for, and does the issue give the fix away. 328 of 430 attempts failed this review.Environment. The repository is fetched at the PR's base commit with no later history. An Opus agent, working in a throwaway container without the issue or the fix, writes the install script; the final image is then rebuilt from that script alone and checked: the working tree is clean at the base commit and no later commit exists in it.Validation. With the hidden tests added, at least one test must fail on the original code and pass with the upstream fix in three separate fresh containers; an empty patch must not pass; flaky tests are dropped; the test run must finish in under 10 minutes.Freeze. Fifty test tasks and ten development tasks were drawn by difficulty with at most three per repository, the manifest was fixed and its hash published, and the comparison to report was declared before any model ran.From 18,132 pull requests to 50 tasks Stage 1, automatic filters (no model involved).
0 5k 10k 15k 20k Python repos found GitHub search: ≥ 1,000 stars, > 200 forks, pushed since 1 Sep 2,985 Repos kept OSI licence, ≥ 60 % Python, active, issue-linked PRs since 1 Sep 300 Merged PRs crawled merged PRs created since 1 Sep 18,132 Issue also after 1 Sep after the issue_before_cutoff filter: 14,106 14,106 Code + test changes after the docs_ci_typing_only filter: 8,758 8,758 Candidate pool after the issue_contains_fix_code filter: 5,450 5,450 Shortlist (≤ 3 per repo) 763 from 275 repos 763 count From 430 attempts to 50 Stage 2: clarity review, environment build and validation.
0 200 400 600 Attempted (ordered list) taken in a frozen, seeded order 430 Passed the clarity review 328 rejected by the clarity review 89 Validated gold fix passes 3/3, empty patch fails, hidden tests fail before the fix 62 Test slice 50 test + 10 dev (dev never reported) 50 tasks Table view Reason Rejected tests check things the issue does not state 161 issue gives the fix away 95 issue underspecified 71 clarity_error 1 None 12 test patch disables tests 7 base:eval error 7 no f2p 5 setup session timeout 3 base:test timeout 2 setup disallowed pytest arg:--timeout-method=thread 1 gold:eval error 1 setup agent gave up 1 flaky f2p 1
What it tests Whether a model, driving a coding agent, can read a real bug report, find the cause in an unfamiliar repository and change the code so that the project's own new tests pass, without breaking the tests that passed before.
Median issue: 299.5 words; median fix: 7 added lines (90th percentile 26). Median 2 hidden test(s) must flip from failing to passing and 29.5 existing tests must keep passing. 35 of 50 tasks come from repositories that appear in no public SWE dataset. How a run is scored Agent: mini-swe-agent with native tool calls, one attempt per task, the container has no network. Budget: 60,000 completion tokens per task; sampling temperature 0.7, top-p 0.95, seed 1234, 8,192 tokens per reply. The agent never sees the hidden tests. Its final diff is applied to a fresh copy of the image and the swebench 5.0.2 grader runs pytest -rA on the test files. Solved = every fail-to-pass test passes and every pass-to-pass test still passes. Integrity The task list is held back so it cannot leak into future training data. The commitment below is the SHA-256 of the exact manifest bytes followed by a secret 32-byte salt. When the slice is retired, the manifest and salt are published and anyone can check that the scores above came from these tasks.
5371c3b8d1caedc134657a17b86cad94fc7fe13d81365e932d068f5e22a9c710
sha256(manifest bytes || 32 raw salt bytes)
Read these results with Contamination-reduced, not contamination-proof. Dates rule out these exact fixes, not similar ones elsewhere.Opus-curated. Claude Opus chose which issues are clear enough and set up the environments, and Anthropic's service saw the selected tasks: treat the slice as semi-private for Anthropic models.Easier than SWE-bench Verified. The median fix is 7 lines and only 6 tasks are hard, so strong models crowd near the top.One pass, 50 tasks. Differences under about 10 tasks between two configs are within noise.Configurations, not just models. 27B hit 39 output truncations and 67 repetition loops at these sampling settings; part of its gap is how it behaves under this harness.Hardware, bandwidth measurements, status rules and protocol →
Runs & Data Every config and run, including failed and crashed ones. Leaderboards are per model; each run page has its command, logs and record.
154 records, no qualified eligible config yet
Target: lab decode ≥ 50 tok/s and lab prefill > 1000 tok/s on the same config, passing §5 stability; kit band top-1 ≥ 0.987, KL ≤ 0.0013.
No qualified headline yet. Best so far
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb · current · x8 · GGUF Q2_0
154 runs in 23 configs: ✓ pass 130 ! fail 17 ✕ crash 7 · leaderboard
99 records, no qualified eligible config yet
Target: 1× RTX 3090, registry speed gate 15 tok/s (lab C1 decode), §5 stability and quality floor (≥ 80% of the reference solve rate). 3-card configs are a separate recipe.
Best 1 card
Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 · exllamav3 · ? · current · x8 · below 15 tok/s: documented experiment, no registry PR
Best 3 cards
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · ik_llama.cpp · GGUF IQ2_M · current · 3× x8 · below 15 tok/s: documented experiment, no registry PR
99 runs in 24 configs: ✓ pass 46 ! fail 46 ✕ crash 7 · leaderboard
24 records, no qualified eligible config yet
Target: 1× RTX 3090, registry speed gate 15 tok/s (lab C1 decode), §5 stability and quality floor (≥ 80% of the reference solve rate). 3-card configs are a separate recipe.
Best 1 card
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 · sglang-exl3 · EXL3 3.00 bpw · current · x8 · registry candidate (if §5 passes)
3 cards: no lab measurements yet.
24 runs in 3 configs: ✓ pass 17 ! fail 7 · leaderboard
Recent runs Funding & Contributions Contributions upstream What more hardware would test Each option targets a limit we measured. Expected gains are hypotheses (mostly simulator estimates) until tested.
Tier Option Measured constraint Hypothesis Success criterion No cost Move the model NVMe to a CPU-attached x4 slot GLM NVMe tier saturates the x2 chipset link (3.05 GB/s mean, ≥ 80 % of ceiling 88 % of the time). Simulated 10.8 tok/s instead of 7.0 at a 55 GiB cap; little effect at 90 GiB, where the CPU binds. GLM NVMe recipe ≥ 10 tok/s at 55 GiB. Medium Workstation platform: PCIe 4.0 x16 per GPU, 8-channel memory (e.g. EPYC) trellis Flash-Next is PCIe-bound at x8; GLM's CPU expert lane is bound by 2-channel DDR4 and 13 threads. trellis Flash-Next towards Sero's x16 measurement on the same image (kit C1 61 tok/s, lab 67.5); GLM 23–27 tok/s (simulated on his EPYC box). trellis Flash-Next agentic floor ≥ 50; GLM ≥ 20 tok/s. Large One 96 GB Blackwell card (RTX PRO 6000) Experts that do not fit in 24 GB of VRAM stream over PCIe or run on the CPU. GLM 36–40 tok/s simulated (a lower bound; the 3090's fixed per-token time was used). Flash-Next experts mostly resident. GLM ≥ 30 tok/s agentic floor; Flash-Next ≥ 100 tok/s on a 3-bit quant with higher quality. Not recommended A 4th RTX 3090 None of the tested single-GPU engines pools VRAM across cards. No gain for these engines. —
Costs to be added by the operator. Simulations: moetier on trace G002 (research/moetier-onboarding.md §3).
Runs & Data › Decisions
0001 — Flash-Next llama.cpp comparison quant (T11) decisions/0001-flashnext-llamacpp-quant.md
0001 — Flash-Next llama.cpp comparison quant (T11)
Decision: use bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f IQ4_XS (97.7 GB) for both the 1-card and the 3-card llama.cpp comparison, on the attested ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312 image. N-gram table (per_layer_token_embd.weight) pinned to CPU with -ot; experts on CPU for the layers that do not fit (--n-cpu-moe N, tuned once per layout), fp16→q8_0 KV only if needed for 128k+.
Why: closest to 4-bit that fits 24 GB VRAM + ~110 GiB RAM on one card (IQ4_XS ≈ 91 GiB); same file serves the 3-card run, so the 1 vs 3 comparison changes only the layout. Q4_K_M (119.6 GB ≈ 111 GiB) would not leave room on 1 card. No MTP (llama.cpp Flash-Next MTP PR #27836 not merged). Not REAP-pruned.
Limits: the kit quality panel speaks the SGLang API only, so this route gets no kit band → it cannot be the Flash-Next headline (SPEC §5); it is a comparison point. Delete after T11 unless it beats trellis.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
IQ4_XS stays the paired 1 vs 3 card file. Q4_K_M only as a single feasibility screen if IQ4_XS is competitive with trellis (disk is the binding constraint).
The 1 vs 3 pair freezes occupied context, KV format, batching and workload; --n-cpu-moe / tensor split per layout are reported, and any KV change is a separate comparison.
Full digest + shard hashes via tools/models.py; tensor placement of per_layer_token_embd.weight verified from the server log.
Eligibility experiment-only (no kit band). Results kept regardless; weights deleted only after manifests exist.
Independent review (0001-flashnext-llamacpp-quant.review.md)
AMEND. Keep IQ4_XS as the primary paired comparison, but replace the categorical Q4_K_M rejection with a memory-budget check. The strongest counter-argument is that the decision compares the entire file against host RAM , despite offloading weights to VRAM.
Q4_K_M is not demonstrably excluded. 119.6 GB ≈ 111.4 GiB . If 18–20 GiB of weights reside on the GPU and their host pages are reclaimable, host weight residency becomes roughly 91–93 GiB , leaving 17–19 GiB within the stated 110 GiB RAM budget. Loading peaks, pinned allocations, KV and scratch could consume that margin; the supplied numbers do not prove they will.
Concrete alternative: use IQ4_XS for T11, with one bounded Q4_K_M feasibility screen at the intended context if the allocation budget supports it. Keep any Q4 results as a separate quant comparison. Do not expand into another tuning grid.
Fitting is not evidence of adequate speed. Dual-channel DDR4-3200 has a theoretical 51.2 GB/s ceiling. At 50 tok/s, that permits only 1.024 GB of DRAM traffic per token . If all 6B active LM parameters required fresh ideal 4-bit reads, that is ~3 GB/token and at most 17 tok/s before overhead. Record CPU offload placement and measured decode; this is a conditional bandwidth screen, not a performance prediction.
“Only the layout changes” needs qualification. Independently tuning --n-cpu-moe N compares each layout’s selected configuration, rather than isolating card-count scaling. Freeze occupied context, KV format, batching and workload across the pair; report both CPU-expert allocations and tensor splits. Treat any KV-format switch as a separate comparison.
Complete provenance before launch. Pin the full image digest already supplied in SPEC §6, not just server-cuda12-b11312. Add the exact GGUF shard list and hashes, immutable launches, and the actual tensor-placement evidence for per_layer_token_embd.weight. The proposed -ot setting remains unverified in this review.
The quality limitation is correctly identified, but applies to registry eligibility too. Without the required kit band, this Flash-Next configuration is experiment-only , even if all registry lab gates pass. Neither IQ4_XS nor a speed win establishes quality equivalence.
Replace “delete unless it beats trellis.” Define the comparison at matched occupied context and cache state, and retain results regardless of outcome. Delete rejected weights only after hashes and immutable references are recorded; retain any finalist under §9. A useful three-card result need not beat a single-GPU engine to justify publication.
Runs & Data › Decisions
0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22) decisions/0002-glm-quant-plan.md
0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)
Source: bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 (made with llama.cpp b11279, mainline-compatible; MTP layers kept at Q4_0). Unsloth GGUFs excluded: not loadable on mainline until re-uploaded.
Layout Quants Size Fit reasoning 1 card (24 GB VRAM + ~115 GiB usable RAM) IQ2_M, Q2_K 120.6 / 125.7 GB (112 / 117 GiB) ~21 GiB on GPU, rest in RAM 1 card, stretch IQ3_XXS 138.8 GB (129 GiB) over RAM by ~10 GiB → mmap paging from NVMe; run once, expect slow, record it 3 cards (72 GB VRAM + RAM) IQ3_XXS, IQ4_XS 138.8 / 177.7 GB IQ4_XS ≈ 165 GiB: ~66 GiB VRAM + ~99 GiB RAM; ~4-bit like every accepted GLM recipe
Order (disk is ~750 GB for everything, so sequential): IQ2_M → Q2_K → (IQ3_XXS) → delete the losing 2-bit → IQ4_XS. ik_llama.cpp + MTP runs on the same files.
Quality judged per SPEC §5 (tools gate, 0 parse failures over ≥100 calls, ≥80 % of reference solve rate in the uncapped eval).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Order: IQ2_M and Q2_K on 1 card → same files on 3 cards → IQ3_XXS on 1 card (tight resident candidate, not presumed paging) and 3 cards → IQ4_XS on 3 cards.
Fit is checked by measured placement at the stated context, per GPU; uneven splits watched.
Nothing is a finalist before the uncapped quality evaluation; deletion only of rejected candidates, after manifests.
MTP on/off is its own variable family; MTP use verified from engine logs and metrics.
Publish tool-call counts and reference solve counts next to every quality verdict.
Independent review (0002-glm-quant-plan.review.md)
AMEND. Keep the candidate quants, but correct the IQ3_XXS fit claim and make deletion depend on qualification. The strongest counter-argument is that weight capacity does not establish useful speed or quality : the spec already estimates one-card GLM at 8–13 tok/s, below the 15 tok/s registry floor.
IQ3_XXS is not automatically a paging experiment. Using the decision’s assumptions: 138.8 GB ≈ 129.3 GiB ; subtracting 21 GiB GPU residency leaves 108.3 GiB in RAM , about 6.7 GiB below the stated 115 GiB usable budget. Treat it as a tight resident-memory candidate. Declare paging only after accounting for context, runtime allocations and actual tensor placement.
All fit claims need a specified context and measured placement. Approximate host-weight budgets are 91.3 GiB for IQ2_M , 96.1 GiB for Q2_K , and 99.5 GiB for three-card IQ4_XS . Their remaining host margins are roughly 23.7, 18.9 and 15.5 GiB respectively. These exclude additional runtime memory. Likewise, allocating 66 GiB across three cards leaves only 2 GiB per card on average ; an uneven split can exhaust one GPU.
Budget bandwidth before promising 15 tok/s. Dual-channel DDR4-3200 has a theoretical maximum of 51.2 GB/s . At 15 tok/s, that permits at most 3.41 GB of DRAM traffic per generated token , with a smaller practical allowance. Measure effective bandwidth, expert placement and decode; VRAM capacity alone cannot show that the one-card route clears this constraint. Also, Q2_K is only 4.2% larger than IQ2_M—kernel efficiency and quality could easily determine the winner.
“~4-bit like accepted recipes” is precedent, not qualification. The spec’s reported sub-4-bit KLD of 0.28–0.38 makes quality a substantial risk. Establish the reference explicitly: exact EXL3 if loadable, otherwise the highest-bit GGUF that fits. If that is IQ4_XS, retain it through all candidate evaluations. A speed winner cannot be called a finalist before the uncapped quality evaluation.
Concrete alternative order: screen IQ2_M and Q2_K on one card, compare those files on three cards, then screen IQ3_XXS on both layouts and IQ4_XS on three cards. Run uncapped quality evaluation for promising configurations, then qualification for survivors. Give one-card IQ3_XXS an ordinary residency check before labelling it “stretch”; make three-card IQ3_XXS an explicit comparison point.
Replace “delete the losing 2-bit” with “delete a rejected candidate.” All four listed quants total 562.8 GB , leaving about 185 GB of the specified 748 GB free before Qwen weights, images and other artifacts. Build the combined inventory first. Preserve hashes and file references before deletion, and retain qualifying finalists under §9; being slower than another quant does not itself establish rejection.
Shared files do not establish shared MTP support. Verify that each pinned engine loads this exact revision and actually uses the retained MTP layer. Compare MTP off/on as a separate variable family; record accepted tokens/s, acceptance rate, settings and quality. Keep locally built ik_llama.cpp results experiment-only until its image provenance becomes eligible.
The decision’s quality summary omits essential acceptance conditions. Require the rolling 60-second decode floor , three qualification runs, cold-cache reset, no OOM/crash/repetition, and all six lab gates. Zero failures in 100 tool calls satisfies the spec but still gives an approximately 3% one-sided 95% upper bound on failure probability. The solve-rate test is also coarse: with four reference solves, all four are required; with five, four suffice. Publish these counts alongside occupied context and exact launch provenance.
Runs & Data › Decisions
0003 — Harness interpretations of SPEC §5 (repetition, parse failures) decisions/0003-harness-interpretations.md
0003 — Harness interpretations of SPEC §5 (repetition, parse failures)
Context: the T06a harness was exercised for 10 min against the operator's daily Qwen3.8-27B (EXL3 3 bpw, SGLang): 73 tool calls, 0 malformed tool calls, but 12 repetition hits — genuine reasoning loops (the same multi-paragraph block 3× inside one reasoning field) on sympy-21612 at 60–83k occupied tokens. Under §5 as written, the operator's everyday model fails stability.
Interpretations implemented by the harness:
Repetition, per field: the 64-token ×3 rule is evaluated within each generated field (reasoning, content, each tool call's arguments), not over the concatenation, because code drafted in reasoning and then emitted in a tool call trips the joined check without being a loop.
Parse failures, strict: besides invalid tool-call JSON and unparsed tool-call markup, every agent format error counts — including a response truncated at max_tokens (8192) with no tool call. Each response counts once; causes are split in detail.
Same-call streak (≥4 identical consecutive tool calls) resets at each task attempt.
Proposed (pending data, operator decides): keep §5 strict for this first pass and record all hits with causes; after T10/T20 produce real data, revisit whether "no repetition loops" should mean "no unrecovered loop" (a loop that ends the response by hitting max_tokens or ends the task) — not changed now.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
SPEC §5 amended: per-field repetition (cross-field diagnostic only), detector frozen for T10/T20, streak resets only on a fresh conversation (each task attempt starts a new mini-swe-agent conversation, so the harness already complies).
Three error categories: parse_failures (malformed tool calls only — the qualification criterion), output_truncations, agent_format_errors, all published.
No relaxation of "no repetition loops" now. Recovery and wasted-token figures to be published as supplementary data; any protocol revision later is prospective and does not re-grade earlier results.
Denominators always shown next to zero-failure claims.
Independent review (0003-harness-interpretations.review.md)
AMEND. Keep the strict no-loop gate, but distinguish detector clarification from changes to eligibility. The strongest objection is that the harness simultaneously narrows repetition detection and broadens parse failures , while claiming to preserve §5.
Per-field repetition is defensible, but requires an explicit spec amendment. “Within one response” currently includes repetitions across fields. Adopt: “Within reasoning, content, or each tool-call argument field, independently, using the served model’s tokenizer.” Retain cross-field matches as diagnostics so the excluded cases remain auditable.
Do not label every agent-format error a tool-call parse failure. Invalid JSON and malformed tool markup qualify; an otherwise valid response exhausted at max_tokens without a tool call is a truncation or agent-protocol failure. Record tool_parse_failure, agent_format_error, and output_truncation separately. If all must disqualify, add that requirement explicitly to §5.
Resetting the same-call streak is appropriate only at an independent task attempt. Specify that a fresh agent conversation resets it; a retry continuing the same conversation does not. Otherwise attempt boundaries can conceal four consecutive identical calls.
The daily-model result does not justify relaxing the gate. Twelve genuine loops establish that this particular screening run fails the agreed criterion. They do not establish that the criterion is unsuitable for the two target models. The observed 60–83k occupancy also falls below the ≥128k requirement for “agentic-usable.”
Reject “unrecovered loops only” as the qualification alternative. Recovery does not erase wasted generation or latency. At exactly the specified decode floors, an 8,192-token response takes approximately 164 seconds for Flash-Next or 546 seconds for GLM , excluding prefill—27% or 91% of a ten-minute run. Publish recovery and wasted-token measurements as supplementary metrics while retaining zero detected loops for qualification.
The parse evidence is preliminary. Seventy-three calls falls short of the ≥100-call minimum. Under an independent binomial assumption, 0/73 failures still gives a one-sided 95% upper failure-rate bound of approximately 4.0% ; 0/100 gives 3.0% . Report the denominator without presenting zero observed failures as strong reliability proof.
Concrete alternative: ratify the field boundary and separate error categories before target comparisons; freeze that detector version for T10/T20; preserve each offending response and count affected responses separately from detector matches. Use later data to propose a prospective protocol revision, with existing results retaining their original qualification status.
Runs & Data › Decisions
0004 — Quarantine CPU core 2 (logical CPUs 2 and 18) decisions/0004-core2-quarantine.md
0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)
Evidence: two hard resets on 2026-10-03 (04:57 near-idle, 06:56 during T10 model load). Each next boot logged an uncorrectable machine check in bank MC5 (Execution Unit) — CPU:18 MC5_STATUS[-|UE|MiscV|-|PCC|TCC|SyndV] and CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV]. CPUs 2 and 18 are SMT siblings of physical core 2. No OOM, no GPU Xid, no kernel panic text: the core faulted and the platform reset. Likely cause: per-core Curve Optimizer/PBO undervolt too aggressive for that core, or core degradation (RMA case if BIOS is stock; BIOS 5101).
Decision (operator, option 2): take CPUs 2 and 18 offline for the whole campaign (resets at reboot; hygiene.sh check refuses to benchmark while they are online). Every record carries host.cpus_online/cpus_offline. llama.cpp and ik_llama launches use 15 threads. Then validate with tools/stability/cpu_check.sh (all-core AVX2, per-core single-thread boost bursts, all-core openssl) and look for new machine checks.
Consequence for results: numbers are for a 15-core 5950X. CPU-bound routes (GLM experts on CPU) are affected ~6 %; GPU/PCIe-bound trellis decode should not be. Any result produced before this decision does not exist (T10 never completed).
Check result (2026-10-03 07:25–07:40)
cpu_check.sh 300 20 180 on the 30 online CPUs: 0 machine checks, no reset (log: logs/cpu_check-core2-offline.log). Not conclusive for the near-idle failure mode; the hygiene check keeps the quarantine enforced for every run.
Revision (2026-10-03 10:30) — quarantine lifted with a tripwire
Operator BIOS check: PBO was Auto (ASUS stock); Curve Optimizer set to all cores, positive 0 (= no offset). So the CPU was effectively at stock when it faulted; a weak core 2 remains the leading hypothesis.
TARGET_CPUS="2 18" cpu_check.sh 300 20 180 with all 32 CPUs online: all-core AVX2, per-core bursts, 5 min sustained on CPU 2 and 18 each, 5 min on/off bursts on each, all-core openssl — 0 machine checks , no reset.
Not conclusive (both crashes came after hours of uptime). Core 2 is back online for full capacity; hygiene.sh check refuses to benchmark and re-offlines CPUs 2,18 if any machine check appears in the current boot; every record carries host.machine_checks_this_boot. llama.cpp/ik_llama launches back to 16 threads.
T10 ran on 15 cores (recorded in its host block). Trellis decode is PCIe-bound; one 16-core re-check is planned.
If it recurs at stock: keep core 2 offline for the campaign; RMA candidate.
Revision (2026-10-04 08:05) — permanent quarantine
Third fatal reset with core 2 online at stock: 2026-10-04 06:14, ~20.5 h into the boot, during the 16-core T10 re-check model load (CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], logged at the 06:54 boot). Same pattern as the 2026-10-03 06:56 crash (Flash-Next load) after a long uptime.
Core 2 stays offline for the rest of the campaign, enforced at boot by local-ai-bench-quarantine.service (tools/box/install.sh) and by hygiene.sh check (CPU_QUARANTINE defaults to "2 18"). All llama.cpp/ik_llama launches use 15 threads. The 16-core T10 re-check is dropped; all results are for a 15-core 5950X.
Stock settings + repeated MC5 UE on one core: AMD RMA candidate (operator's call).
Runs & Data › Decisions
0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe) decisions/0005-ik-llama-glm-tool-parsing.md
0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)
Finding (T22b, IQ2_M, 1 card): the model emits well-formed GLM tool calls (<tool_call>bash<arg_key>command</arg_key><arg_value>…</arg_value></tool_call>) but ik_llama's server returns them as plain content with empty tool_calls: lab tools gate fails; stability run 0 tool calls / 47 parse failures (unparsed_markup). ik_llama builds tool parsers from the chat template (common/chat-auto-parser*); mainline llama.cpp b11312 parses the same GGUF correctly (tools gate passed in T20b). No matching ik_llama issue/PR found (search 2026-10-04).
Speed is unaffected and recorded: decode 10.0 tok/s (mainline 9.6), prefill 151 lab / 203 TTFT-8k (mainline ~80).
Decision: keep the ik_llama route as speed evidence only (experiment-only, fails §5 tools); do not patch ik_llama in this campaign — single-card GLM is ~10 tok/s on every route, below the 15 tok/s gate, so tool parsing would not change the outcome. Phase 3 option: upstream bug report with the captured response (artifacts/runs/…ik_llama…stability/proxy/parse_failures/).
3-card follow-up (2026-10-04, GPU2-first order, decision 0006)
No MTP, --fit-margin 4096: lab decode 13.4 / 14.0, agentic 13.3 tok/s steady (best GLM decode on this box), prefill 185–283; tool calls still unparsed (68 agent-format errors), tools gate fails.
MTP, --fit-margin 7168 (4096 OOMed creating the MTP context): lab decode 10.2 (acceptance 51 %), agentic 15.6 whole-run before the server crashed mid-run. The extra margin for the draft context pushes experts to host RAM.
Route parked: experiment-only speed evidence. The most interesting number for 0xSero is 13–14 tok/s without MTP; a tool-parser fix in ik_llama plus an attested image would be needed before it could be a 3-card candidate.
Runs & Data › Decisions
0006 — PCIe lane layout: what can be reconfigured, and what it buys decisions/0006-pcie-layout.md
0006 — PCIe lane layout: what can be reconfigured, and what it buys
Operator question (2026-10-04): can the PCIe lanes be reconfigured to get better model performance?
Measured topology (lspci, nvidia-smi, T05a)
Device Attach Link Measured H2D GPU0 3090 04:00.0 X570 chipset, behind a Gen4 x4 uplink shared with the KC3000 1 TB NVMe (root + all model files), NIC, BMC, SATA, USB Gen4 x8 to the chipset 6.0 GB/s GPU1 3090 0b:00.0 CPU lanes, slot 2 (x8/x8 split) — drives the display Gen4 x8 not measured (display) GPU2 3090 0c:00.0 CPU lanes, slot 1 (x8/x8 split) Gen4 x8 13.5 GB/s 990 EVO Plus 4 TB CPU lanes (M.2_1, Gen4 x4) unused: holds another OS install
Already optimal and not a lever: every link trains at Gen4 (16 GT/s); Resizable BAR is on (32 GB BAR1); ASPM is disabled on every GPU link. AM4 has 24 usable CPU lanes (16 slots + 4 M.2 + 4 chipset uplink); bifurcation or risers cannot add lanes, so the only real choices are which card sits where and how many cards share the 16 slot lanes .
Where PCIe bandwidth limits each route (evidence from our runs)
Flash-Next trellis (experts streamed from host RAM to one GPU): decode is PCIe-bound. 0xSero's data: ~26 GB/s → 62.7 tok/s, 14–16 GB/s → 53.2. Ours at 13.5 GB/s: lab 49–55, kit C1 46.9, agentic 43. x16 should add ~15–25 %.
llama.cpp/ik_llama prefill with experts in RAM is PCIe-bound (CPU-resident weights are copied to a GPU for large batches). Evidence: on GPU2, raising -b/-ub 512 → 2048 lifted GLM IQ2_M prefill 85 → 200 tok/s (each copy serves 4× more tokens). And 3 cards prefilled slower than 1 card (158 vs 200), consistent with the copy going to CUDA0 = GPU0, the 6 GB/s chipset card.
llama.cpp decode with experts on the CPU is DDR4-bound, not PCIe-bound (GLM ~9.5 tok/s on 1 card, 12.3 on 3 cards, from less RAM-resident weight). PCIe changes nothing here.
Options
# Change Cost Expected effect S1 3-card llama.cpp with the CPU-attached GPU2 as CUDA0 (CUDA_VISIBLE_DEVICES=2,1,0 in the container) none 3-card prefill back to ≥ 1-card level (~200+), no decode change S2 Larger ubatch (-b/-ub 4096) on llama.cpp routes none (more VRAM for compute buffers) prefill up to ~1.5–2× again; watch VRAM (the IQ2_M 3-card run already OOMed on CUDA0) P1 One card alone in slot 1 at x16 (T12, already planned) operator moves cards Flash-Next decode +15–25 % (the only route to ≥ 50 headline at full context); llama.cpp prefill ~2× vs x8; GLM exact mode at the bandwidth its 12.6 tok/s reference used P2 Plug the monitor into GPU0 (chipset card) instead of GPU1 move one cable GPU1 + GPU2 (both CPU x8) free for 2-card runs without going headless; desktop stays usable during single/2-card work P3 Model files on the CPU-attached 990 EVO Plus instead of the chipset KC3000 needs the operator's OK (other OS lives there) removes model loads and Flash-Next n-gram NVMe reads from the chipset uplink that GPU0 also uses; small effect except for 3-card runs with GPU0 P4 2 cards on CPU lanes (x8/x8), third card removed or idle none (GPU1+GPU2) avoids the 6 GB/s card; less VRAM (48 GB) so more experts in RAM
Decision (proposed)
Run S1 and S2 now in the current layout (software only, attributable, one variable family each), on GLM IQ2_M: 3-card GPU2-first at ub2048 (vs the existing GPU0-first record), then 1-card GPU2 at ub4096 (vs ub2048).
Keep P1 (T12) as the main hardware lever; in the x16 session also rerun the best GLM llama.cpp 1-card config so the x8→x16 effect on prefill is measured directly.
Recommend P2 to the operator at the T12 visit (same visit, one cable). P3 only if the operator wants to free the 990 EVO Plus; not worth touching another OS for this campaign.
Publish this as a site methodology section with the before/after records.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted, plus operator rule
Bandwidth effects are hypotheses until measured. Expected-gain numbers above are estimates, not claims; x16 is the principal hardware hypothesis for Flash-Next, not a guarantee of qualification.
S1 is a fresh paired comparison, both headless, identical except card order: IQ2_M 3-card ub2048 with a 2 GiB --fit-target margin (the default-margin run OOMed on CUDA0 after 62 min), GPU0-first vs GPU2-first (CUDA_VISIBLE_DEVICES=2,1,0; the container sees the cards in PCI order). Every TTFT step now logs per-GPU PCIe rx/tx (nvidia-smi dmon -s t) to show which card receives the weight copies. Decode changes are reported, not assumed.
S2 : 1-card GPU2 ub2048 vs ub4096, both at 15 threads (the old ub2048 record ran 16 threads), peak VRAM and OOMs recorded.
P1 : rerun the best 1-card GLM llama.cpp config at x16 with the same bandwidth procedure (x8 control exists).
Operator rule (2026-10-04): GPU0 is used last. Single-card runs use GPU2; 2-card runs GPU2+GPU1; 3-card runs order the cards GPU2, GPU1, GPU0. Pending S1 evidence, all new multi-card launches use the GPU2-first order.
P2 offered to the operator at the T12 visit; P3 deferred (would need a §9 exception and touches another OS).
S1 result (2026-10-04 12:15–13:05 AEST) — card order, GLM IQ2_M 3-card ub2048, 2 GiB fit margin, headless
GPU0 first (control) GPU2 first (CUDA_VISIBLE_DEVICES=2,1,0) change TTFT prefill 8k / 32k 159 / 169 tok/s 240 / 242 tok/s +51 % / +43 % lab prefill 153 tok/s 218 tok/s +43 % lab decode C1 12.2 tok/s 11.8 tok/s −3 % PCIe rx during TTFT (avg / peak) GPU0 3.9 / 6.5 GB/s (link-saturated), others ~0.3 GPU2 5.7 / 14.3 GB/s, others ~0.5
The host→device weight copies for prefill go to CUDA0; putting a CPU-lane card there lifts prefill ~1.5×. Decode is within noise of unchanged (slightly lower). The GPU2-first config also finished a clean 10-min agentic run (11.3 tok/s, 24 tool calls, 0 parse failures, 0 loops) and passed the post-run gates without the OOM that hit the default margin. Rule confirmed: GPU0 last.
S2 result (2026-10-04 13:05–14:23 AEST) — ubatch, GLM IQ2_M 1 card GPU2, 15 threads
ub2048 (control) ub4096 change lab prefill 201 tok/s 288 tok/s +43 % TTFT prefill 8k / 32k 213 / 222 321 / 316 +51 % / +42 % lab decode C1 9.5 9.6 unchanged VRAM at ready 21.9 GB 22.9 GB +1 GB, no OOM in a 10-min agentic run
The 15-thread control matches the earlier 16-thread ub2048 record (9.5 / 200), so the core-2 quarantine costs nothing measurable on this route. Gain > 3 %, so the search continues: ub8192 on 1 card, and ub4096 on the GPU2-first 3-card config.
S2 step 2 (2026-10-04 16:13–16:30 AEST) — ub8192, 1 card GPU2
lab prefill 370 (+28 % vs ub4096), TTFT 384 / 404, but lab decode 9.1 (−5 %): the larger compute buffer makes --fit keep more experts in host RAM (VRAM at ready 22.2 GB). Prefill and decode now trade off, so the batching search stops here: ub4096 is the balanced setting (decode unchanged, prefill +43–51 %); ub8192 is the prefill-maximising variant, published as such.
3-card GPU2-first, ub2048 → ub4096 (16:30–16:58): lab prefill 218 → 258 (+18 %), TTFT 240 → 298 (+24 %), lab decode 11.8 → 11.2 (−5 %). Same prefill/decode trade-off as 1 card at ub8192; both published, no further steps.
Independent review (0006-pcie-layout.review.md)
AMEND. Proceed with S1, a bounded S2 test, and T12, but describe the bottlenecks and speedups as hypotheses. The strongest counter-argument is that the current runs change batching, weight placement, and GPU count; they do not isolate PCIe bandwidth.
Flash-Next’s +15–25% estimate is plausible; “PCIe-bound” overstates the evidence. The reference gain is only 17.9% despite bandwidth rising from 14–16 to 26 GB/s. A simple fixed-work Amdahl model attributes roughly 33–39% of baseline time to transfers, predicting 20–25% from doubling bandwidth. That model remains unverified across machines and contexts.
x16 does not establish qualification. Applying +15–25% to the measured agentic 43 tok/s gives 49.5–53.8 tok/s , and averages cannot prove every required rolling window clears 50. Replace “the only route to ≥50 headline” with “the principal hardware hypothesis”; qualification still requires the same config to pass occupied-context, prefill, quality, and stability requirements.
The prefill explanation is not established. Increasing batch size fourfold produced 2.35× throughput, from 85 to 200 tok/s. Transfer amortization, better GPU utilization, and execution overhead can all explain this. Likewise, 158 versus 200 tok/s across three versus one GPU does not identify CUDA0 as the cause. Replace “PCIe-bound” and “~2× at x16” with measured observations and an explicit hypothesis.
Make S1 a fresh paired comparison. Run GPU0-first and GPU2-first at ub2048 with identical weights, offload settings, tensor split, context, and cache state; both must be headless. Select devices by UUID , record their logical CUDA mapping, and verify which GPU actually receives the expert transfers. Reordering devices alone does not demonstrate the claimed routing change. Report decode changes rather than assuming none.
Bound S2 by memory and measurement. Compare one-card GPU2 ub2048 with ub4096 while holding the other settings fixed; record peak VRAM and OOMs. Remove the promised “1.5–2× again.” If both batch and ubatch change, identify the comparison as a batching-family change without attributing its effect specifically to ubatch.
Keep P1, with a matching x8 control. Run the exact same best one-card GLM configuration before and after the move, alongside bandwidth measurements under a defined, identical procedure. The 12.6 tok/s exact-mode reference already used x16 and remains 16% below the 15 tok/s floor ; matching its link does not imply registry eligibility.
P2 is useful, subject to the display rule. Moving the display to GPU0 enables display-free GPU1/GPU2 comparisons. Three-card tests still require logout and display: off; the x16 benchmark must also use a GPU that is not driving the display.
Defer P3 and public publication as proposed, but clarify scope. Storage contention has not been measured, and changing active-model storage would require an explicit exception to §9. Put the methodology and every before/after result in the private preview now ; public publication remains Phase 3 after operator and log review.
Runs & Data › Decisions
0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs decisions/0007-glm-current-layout-finalists.md
0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs
Facts (all current-layout GLM records, 2026-10-03/04)
Route Cards Lab decode Lab prefill 10-min run (tool calls / parse failures / loops / solved) llama.cpp IQ2_M ub4096 1 9.6 288 27 / 0 / 1 / 1 of 2 llama.cpp IQ2_M ub2048, GPU2 first 3 11.8 218 24 / 0 / 0 / 1 of 2 llama.cpp IQ2_M ub2048, GPU0 first 3 12.3 158 26 / 0 / 0 / 1 of 2 (server later OOMed) llama.cpp IQ3_XXS ub2048 3 10.4 132 22 / 0 / 1 / 1 of 2 llama.cpp IQ4_XS ub2048, GPU2 first 3 8.3 156 25 / 0 / 0 / 1 of 2 llama.cpp Q2_K / IQ3_XXS 1 9.2 / 8.3 78 / 65 0–1 solved ik_llama IQ2_M (± MTP) 1 8.2–10.0 (MTP agentic ~12.6) ~150 0 tool calls: GLM tool calls not parsed (0005)
Every route is below the 15 tok/s registry gate on 1 card and on 3 cards (best 12.3). Decode is DDR4-bound: more cards only help by holding more experts in VRAM. Pending in the queue: IQ2_M 1-card ub8192 and 3-card ub4096 (prefill-only), ik_llama 3-card ± MTP (speed evidence only).
Proposal
No current-layout GLM finalist. §5 qualification requires decode ≥ 15 tok/s in every window, so the 3 × 10-min qualification runs would fail by construction. T25a records "no finalist: below the floor" for every current-layout config instead of running them. All configs stay published as documented experiments (§6 Phase 2).
One uncapped quality evaluation, of the best mainline weights, in the x16 session. Solve rate depends on weights, engine and sampling, not on the PCIe layout; the layout only changes how long the evaluation takes. So the T07 candidate evaluation of llama.cpp IQ2_M runs on the single x16 card next to the T23 reference (exact mode, or the highest-bit GGUF that fits if exact mode cannot load), with the identical task list, sampling and 60k-token budget. Each evaluation is estimated at 4–7 h (measured ~3–4k completion tokens per 10 min).
No uncapped evaluation for IQ3_XXS / IQ4_XS / Q2_K / ik_llama. They are slower than IQ2_M and cannot become candidates; their 10-min screening solve counts are published labelled "screening only". IQ4_XS only fits on 3 cards, so measuring it would cost ~6 h of current-layout time for an informational number.
Consequence: T12 (card move) can happen as soon as the current queue finishes (~3.5 h), not after ~1 day of current-layout quality runs. After the queue, IQ3_XXS and IQ4_XS are marked rejected and deleted (manifests kept) to make room for the x16 weights (Flash-Next 2.05/4.05 bpw: 63/108 GB; GLM exact EXL3: 125 GB).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Below-floor is measured, not inferred from lab C1: every current-layout llama.cpp config with a 10-min run has a measured rolling 60-s generation-only minimum below 15 (IQ2_M 3-card 11.0–11.4, IQ2_M 1-card 8.8–9.1, IQ3_XXS 3-card 9.9, IQ4_XS 3-card 7.8). Configs without a stability run (prefill-only S1/S2 controls, ub8192) are recorded as "qualification skipped: subfloor lab C1; rolling floor not measured". T25a closes with no current-layout finalist once the pending queue (ik_llama 3-card ± MTP, ub follow-ups) is in; ik_llama stays experiment-only (0005).
Quality evaluation is conditional. At x16: measure IQ2_M 1-card first; only a config that qualifies (§5 incl. the rolling floor, ≥ 100 cumulative tool calls, post-run gates) gets the uncapped T07 candidate evaluation. An optional quality study of a non-qualifying config is labelled informational. Quality results apply to the tested configuration; layout independence is not assumed.
Runtime estimate corrected: the 60k budget is per task; worst case 2.5–3.3 h per task, 12–33 h per evaluation (less when tasks submit early). Budget accordingly; this is another reason to evaluate qualifiers only.
Labels: IQ4_XS, IQ3_XXS (3-card), Q2_K: "rejected for speed, quality unmeasured". IQ2_M 3-card GPU0-first: OOM after 62 min (default fit margin). IQ2_M/IQ3_XXS repetition hits are published with the run.
Storage: keep IQ2_M (x16 rerun) and IQ3_XXS (highest-bit GGUF that fits 1 card = fallback reference if exact mode cannot load); delete IQ4_XS after the queue, manifests and hashes kept for reacquisition.
Independent review (0007-glm-current-layout-finalists.review.md)
AMEND. Finish the queue and move to x16 without spending a day on current-layout quality runs. The strongest objection is that the proposal treats lab C1 as proof of failing a different metric , then schedules a “candidate” quality evaluation for a configuration that has not qualified.
“Fail by construction” is unsupported. Lab decode measures the first 30 seconds; §5 requires generation-only decode over rolling 60-second windows, ignoring the first minute. Best C1, 12.3 tok/s, is 18% below the floor and needs 22% uplift to reach 15. That warrants deprioritization, but not a claimed measurement of the floor. Use existing screening traces to establish a failing window; otherwise record “qualification skipped: subfloor lab C1; rolling-floor eligibility unverified.”
Make x16 quality evaluation conditional. §5 says “Finalists only.” First measure IQ2_M on x16; if promising, complete qualification, including ≥100 cumulative tool calls and all six post-run gates. Run the uncapped comparison only if it qualifies. If it remains below 15, finish it as a documented experiment under Phase 2. Any optional quality study should be explicitly labelled informational.
Layout independence is a hypothesis, not a guarantee. Fixed weights and token budgets remove much of the speed bias, but device placement can change numerical execution and sampled trajectories; timeouts, OOMs and truncation can also affect outcomes. The x16 evaluation directly supports the tested x16 configuration . Transfer its quality result to another layout only with matching engine, settings and evidence of equivalent behavior.
IQ2_M is the speed leader, not established “best weights.” The 10-minute results cannot rank quantization quality. IQ4_XS could be slower yet substantially more accurate—a relevant possibility given §12’s sub-4-bit quality risk. Skipping its uncapped evaluation is reasonable because it has no demonstrated eligibility; label it rejected for speed , with quality unmeasured.
The runtime estimate needs its budget scope stated. At 3–4k completion tokens per 10 minutes, exhausting 60k tokens per task takes 2.5–3.3 hours per task. Across 5–10 tasks, that is 12.5–33 hours per evaluation , or 25–67 hours for candidate plus reference , assuming similar throughput. Early completion can reduce this. If 60k means an aggregate budget, specify the per-task allocation; “4–7 hours” currently lacks a clear basis.
Keep T25a provisional until the queue finishes. Pending ik_llama 3-card/MTP measurements could change the speed ranking, although parsing and image eligibility remain blockers. Existing IQ2_M repetition and GPU0-first OOM evidence also need explicit failure labels. Faster decode alone does not make these configurations finalists.
Treat “DDR4-bound” as a working diagnosis. The table does not isolate memory bandwidth from PCIe, placement, batching or engine overhead. The one-card 9.6 tok/s result needs 56% uplift to reach 15, so an x16 rescue is uncertain. Measure it before committing to expensive quality work.
Concrete replacement: finish pending measurements → close current-layout records with measured failures or justified skips → move cards → measure and qualify x16 routes → evaluate qualifying candidates against the specified reference. Delete rejected weights after preserving hashes and immutable references, while reserving or budgeting reacquisition of the highest-bit GGUF fallback. Keep results private until Phase 3.
Runs & Data › Decisions
0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate decisions/0008-flashnext-3card-llamacpp.md
0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate
Evidence (T11, 2026-10-04 15:38 AEST, IQ4_XS, GPU2,GPU1,GPU0 order, 2 GiB fit margin, -b 2048 / -ub 512 defaults)
Lab: all six gates pass , decode C1 57.1 / 56.9 tok/s (before / after the agentic run), lab prefill 608 / 605.
TTFT prefill: 987 tok/s at 8k, 852 at 32k.
10-min agentic run: 70 tool calls, 0 parse failures, 0 loops, 1 of 2 solved, whole-run decode 45.4, lowest 60-s window 38.6 (fails the 50 floor; decode falls as the context grows).
VRAM at ready 65.5 GB of 72; weights 97.7 GB, so ~30+ GB (incl. the n-gram embedding table) stays in host RAM.
For comparison, the 1-card trellis/SGLang baseline at x8: lab 49–55, kit C1 46.9, agentic 43.
Per SPEC §6 a 3-card config is a separate, clearly labelled recipe; llama.cpp routes are judged by the registry lab and §5 (the kit's SGLang-only panel cannot score a GGUF, so it is "kit band not measurable").
Proposal (current layout, before T12; one variable family per step, stop rule 3 × < 3 %)
Re-download IQ4_XS (deleted at 16:13 per plan; manifest kept, 97.7 GB, ~15 min).
Prefill : -b/-ub 2048, then 4096 (GLM showed +135 % then +43 %; target lab prefill > 1000).
Decode floor : the floor fails late in the run as context grows. One quant step down to IQ3_M (93.2 GB) or IQ3_XXS (88.0 GB) puts more experts in VRAM; run the better prefill setting from step 2 on it. Quality cost is unmeasured by the kit for GGUF; report solve counts and parse failures, label "quality: screening only".
If a config reaches lab decode ≥ 50, lab prefill > 1000 and holds the 50 floor in a 10-min run, it becomes a 3-card finalist: §5 qualification (3 runs incl. cold and max context, ≥ 100 tool calls) before T12.
Disk: the x16 downloads (296 GB) wait until this route is done and its rejected GGUFs are deleted.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Route is experiment-only unless an accepted scoring path establishes the Flash-Next kit band for a GGUF; kit band recorded as unavailable, registry/headline eligibility unestablished. The T11 config is a failed screening config (floor −29.5 %, lab prefill −65 % vs targets), reported per constraint.
Batching steps, predeclared, judged by official lab prefill (decode and VRAM regressions checked): A -b 2048 -ub 2048, B -b 4096 -ub 4096, both vs the existing -b 2048 -ub 512 record. (GLM evidence says the ubatch sets the host→GPU copy amortisation, so b and ub move together.)
Before any quant change: decode vs occupied context from the existing T11 agentic trace (per-request generation speed against prompt tokens). IQ3_M is screened only if that shows a residency/transfer limit, not attention cost; its step is judged by the minimum rolling-window decode at matched occupancy. Expert/n-gram placement and memory use recorded from the server log.
Qualification only for a speed-passing config, naming its maximum occupied context (≥ 4k headroom), cold resets, post-run gates, ≥ 100 tool calls cumulative. Phase timebox kept; x16 is not deferred beyond this route's steps.
Decode vs occupied context (T11 3-card agentic trace, 55 requests, generation-only tok/s per request)
prompt tokens per-request decode 1.5k–6k (django task) 35–51, typical ~42 6k–20k 30–49, typical ~40 20k–31k 30–41, typical ~38 31k–39k 28–38, typical ~33
Two separate gaps to the 50 floor: a base gap (~42 tok/s already at 2–6k tokens, vs lab C1 57 measured over one long generation; agentic requests are short, 70–600 tokens) and a context slope of roughly −20 % from 5k to 39k. One quant step (IQ3_M, −4.6 % of bytes; IQ3_XXS −9.9 %) cannot close a 25–30 % gap even if every saved byte moved experts to VRAM. IQ3_M step dropped ; the predeclared batching steps still run (prefill evidence, cheap). After them the route closes as experiment-only and its GGUF is deleted for the x16 weights.
Batching result (2026-10-04 18:00–18:23 AEST) — route closed
-b / -ub lab decode C1 lab prefill TTFT 8k / 32k 2048 / 512 (T11 record) 57.1 608 987 / 852 2048 / 2048 44.3 (−22 %) 717 (+18 %) 1067 / 931 4096 / 4096 35.0 (−39 %) 713 980 / 881
Larger ubatches buy little prefill and cost a lot of decode: the bigger compute buffers make --fit move experts to host RAM. No setting reaches lab prefill > 1000 while keeping decode ≥ 50, and the agentic floor already failed at the default. Route closed: experiment-only , best config = the default-batch T11 record (all six gates, lab 57.1, agentic floor 38.6). GGUF deleted for the x16 weights (manifest kept).
Independent review (0008-flashnext-3card-llamacpp.review.md)
AMEND — retain this as a separate 3-card experiment, but remove the promised path to registry eligibility until quality is measurable. The strongest counter-argument is contractual: §5 explicitly requires the Flash-Next kit band for both the headline and registry PR. “Kit band not measurable” cannot establish a pass. §12’s lab-only tooling note creates an ambiguity; it does not explicitly waive that requirement.
T11 is a failed screening config despite passing all six lab gates. Its worst window needs 29.5% improvement to reach 50 tok/s; lab prefill needs approximately 65% improvement to exceed 1000. Lab C1 at 56.9–57.1 does not compensate for either failure. Report these as separate constraints.
The prefill experiment is reasonable; the GLM gains are insufficient evidence for its expected payoff. Specify exact settings rather than “-b/-ub 2048, then 4096.” A concrete sequence is baseline -b 2048 -ub 512, then -b 4096 -ub 512, then -b 4096 -ub 1024, measuring identical occupied contexts and fresh prompts. Select using official lab prefill, and check for decode or memory regressions.
The quantization hypothesis needs a residency measurement. IQ3_M saves 4.5 GB (4.6%) ; IQ3_XXS saves 9.7 GB (9.9%) . Against roughly 30 GB remaining on the host, those represent at most approximately 15% / 32% reductions , assuming every saved byte reduces host residency. They do not prove that more active experts become GPU-resident or that transfers shrink proportionally. Record expert placement, n-gram placement, and actual memory use.
“Decode falls as context grows” establishes correlation, not the bottleneck. Before changing quant, report generation speed against occupied context, cache state, and GPU placement. Repeat controlled requests at representative early and late occupancies using IQ4_XS. Otherwise a smaller quant could obscure a context-dependent attention or transfer limitation.
Solve counts cannot replace the required quality comparison. One of two solved tasks is weak screening evidence, especially under a wall-clock cap where faster configurations receive more generation opportunities. Keep “quality: screening only,” but explicitly mark the kit band unavailable and registry/headline eligibility unestablished for every GGUF lacking valid panel scores.
Qualification must name the context it qualifies. Step 4 needs a claimed maximum occupied context, ≥4k generation headroom, the defined cold-cache resets, and post-run lab gates for each qualification run. “Agentic-usable” additionally requires ≥128k occupied tokens. Count ≥100 tool calls across screening, qualification, and any extra runs, as specified; publish truncations and agent-format errors too.
Make the search bounded and the stop rule meaningful. Predeclare which metric determines each step’s “gain”: official lab prefill for batching, minimum rolling-window decode at matched occupancy for quantization. A gain in one metric cannot excuse losing another acceptance criterion. Keep the phase’s 2–3-day timebox; disk sequencing alone should not defer x16 indefinitely.
Concrete revised decision: restore pinned IQ4_XS; run the two explicit batching changes; establish the context-dependent slowdown; screen IQ3_M, then IQ3_XXS only if justified. Run full stability qualification only on a speed-passing configuration. Unless the existing accepted scoring path establishes the kit band, publish the route as experiment-only , preserve its records, delete rejected weights under §9, and proceed to x16. Every 3-card record must include GPU UUIDs and display: off.
Runs & Data › Decisions
0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision) decisions/0009-no-x16-session.md
0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)
Operator, 2026-10-04 19:15 AEST: skip the x16 test; the expected difference is judged marginal. T12 (card move) and T05b are cancelled. The operator's call stands; the estimate it overrides is recorded for the reader: 0xSero's reference gives +17.9 % decode for 14–16 → ~26 GB/s, and our x8 baseline (lab C1 49–55, kit C1 46.9, agentic 43, floor not held) sits just under the 50 tok/s target, so reaching it now depends on tuning at x8.
Re-plan (single card = GPU2, CPU-lane x8, 13.5 GB/s; GPU1 measured identical)
T13 PR #144 rerun at x8 against image f77c72f6 (lab, kit sweep with early exit off, TTFT, panel, 10-min run). Proof for 0xSero is "x8 on this box"; whether he accepts it is asked in Phase 3.
T15 brought forward: the main recipe on EXL3 2.05 and 4.05 bpw, only the weights changed. At x8 the expert bytes per token are the bottleneck, so 2.05 bpw is the most likely route to ≥ 50 sustained; the kit panel decides whether it stays in the quality band (required for the headline and registry PR). 4.05 bpw is the quality anchor.
T23 GLM exact mode on GPU2 at x8 (route test + quality reference; may not load: ~119 GB pinned on ~125 GiB).
T14a/T14b grid on the best quant from 1–2 (expert cache GB × context × KV format; then prefill chunk and n-gram tier), designed as its own decision once 1–3 are in. T16 qualification and headline follow on the winner.
GLM IQ2_M x16 rerun dropped; IQ3_XXS kept as the fallback reference.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Recorded as a deliberate restriction of the search, not evidence that x16 is marginal. Gap to target at x8: kit C1 46.9 → 50 needs +6.6 %; agentic 43 → 50 needs +16.3 %; there is currently no qualifying Flash-Next result.
Order: T13 at 3.05 → matched 2.05 / 4.05 screens (same launch, same context) → kit panel as a filter (top-1 ≥ 0.987, KL ≤ 0.0013) before any headline time → bounded tuning of in-band candidates, 3.05 included . Out-of-band results are published with kit eligibility, never as the registry path. "Bytes per token" is a hypothesis; transfer/cache evidence is recorded where the engine exposes it.
Acceptance unchanged: every eligible rolling 60-s window ≥ 50 and lab prefill > 1000 on the same config at the claimed occupied context. If no x8 config meets speed and quality together, that outcome is published as is.
T23 at x8: exact mode may be the GLM quality reference even below 15 tok/s (not a recipe); peak host memory and any load failure recorded (~110.8 GiB pinned vs ~125 GiB MemTotal). If it cannot load, the reference is the highest-bit GGUF that actually fits one card here, determined by a load test, not assumed to be IQ3_XXS.
Timebox and the 3 × < 3 % stop rule stay. Cancelled tickets (T12, T05b, x16 parts of T13/T14/T23) are recorded as skipped with the operator's reason.
Independent review (0009-no-x16-session.review.md)
AMEND. Respect the operator’s cancellation of x16, but describe it as a deliberate restriction of the search—not an evidence-backed conclusion that the gain is marginal. The strongest counter-argument is that the cited 17.9% uplift could change eligibility , whereas lower quantization risks losing the required quality band.
The target gap is material and measurable. Kit C1 46.9 → 50 requires +6.6% ; agentic average 43 → 50 requires +16.3% , and reaching that average would still not prove the rolling floor. Applying the reference uplift illustratively gives 55.3 and 50.7 tok/s , respectively. These are scenarios, not predictions for this machine or workload.
Retain 3.05 bpw as a tuning candidate. The proposed sequence risks selecting an out-of-band 2.05 model or a slower 4.05 model before exploring the original quant. Concrete alternative: T13 at 3.05 → matched 2.05/4.05 screens → quality filter → bounded tuning of qualifying candidates, including 3.05 .
Treat “expert bytes per token are the bottleneck” as a hypothesis. Neither the bandwidth measurement nor the quoted speeds establishes how much decode time is exposed transfer, compute, synchronization, or cache misses. Moving 3.05 → 2.05 bpw reduces nominal weight bits by 32.8% ; under a purely weight-transfer-limited model that suggests about 49% more throughput. Metadata, cache residency and other costs can substantially change that result. Record transfer/cache evidence where available before assigning causality.
Use the panel early as a search filter. Require top-1 ≥ 0.987 and KL ≤ 0.0013 before spending headline qualification time on a quant. Publish outside-band results and retain their kit eligibility, but do not make them the sole path toward the registry target. Keep occupied context and other settings matched for the initial weight comparisons.
Preserve the actual acceptance test. Lab C1 over 30 seconds, a kit score, and a whole-run agentic average cannot establish a sustained floor. The winner needs every eligible rolling 60-second window ≥ 50 , all qualification requirements, and lab prefill > 1000 on that same config , at the claimed occupied context. The existing “floor not held” evidence means there is currently no qualifying Flash-Next result.
T23 is useful at x8, but distinguish reference eligibility from recipe eligibility. Exact mode may serve as the GLM quality reference even below 15 tok/s ; it then remains ineligible for a registry recipe. 119 decimal GB ≈ 110.8 GiB , leaving roughly 14.2 GiB against the stated 125 GiB MemTotal before runtime and system costs. That is plausible but unproven headroom; record peak host memory and any load failure.
IQ3_XXS cannot automatically become the GLM quality reference. The spec requires the highest-bit GGUF that fits when exact mode cannot load. Keep IQ3_XXS as a performance fallback, but separately determine the fitting quality reference and run the identical-task, identical-budget, uncapped comparison required by §5.
Make the amendment auditable and bounded. Update §6, §11 and the x8 risk discussion; record cancelled tests as skipped with the operator’s reason. Keep the phase timebox and diminishing-return stop rule. If no x8 config meets speed and quality together, publish that outcome explicitly rather than relaxing acceptance. The revised deliverable is the best qualifying result found within the x8 search .
Runs & Data › Decisions
0010 — CPU boost off for the rest of the campaign decisions/0010-cpu-boost-off.md
0010 — CPU boost off for the rest of the campaign
Evidence (2026-10-04 19:48 AEST): hard reset during the T13 Flash-Next SGLang model load with core 2 already offline. The next boot decoded CPU:1 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], "uncorrected error caused a data fabric sync flood" — the same bank, status and syndrome as the core-2 faults (0004), now on core 1 . Tally of fatal resets: 5, of which 3 during a Flash-Next SGLang load (expert repacking: heavy multi-threaded host work) and 2 after long uptime. Temperatures at idle are normal (Tctl ~56–60 °C). A second core failing the same way points to the CPU (or its voltage under boost transients) rather than one bad core.
Decision: disable Precision Boost from Linux (/sys/devices/system/cpu/cpufreq/boost = 0, all cores capped at the 3.4 GHz base clock, lower voltage), applied at boot by local-ai-bench-quarantine.service and immediately on 2026-10-04 20:12. Core 2 stays offline. Every record now carries host.cpu_boost and host.cpu_max_mhz.
Consequence: CPU-bound work (GLM experts on CPU, model loads) gets slower; PCIe-bound trellis decode should not. Records before 20:12 ran with boost on and are labelled by that field's absence (= boost on). If faults continue with boost off, the CPU is not usable for unattended runs and the remaining work pauses for the operator (RMA).
Runs & Data › Decisions
0011 — Flash-Next tuning at x8 (T14): grow the expert cache decisions/0011-flashnext-x8-tuning-grid.md
0011 — Flash-Next tuning at x8 (T14): grow the expert cache
Evidence
T15 is not runnable: the offload engine's HostExpertStore only supports 3-bit experts (offload_store.py: if bits != 3: raise NotImplementedError), so 2.05 and 4.05 bpw crash at load in both images. Only 3.05 bpw remains (the CPU-expert fallback, SGLANG_EXL3_MOE_OFFLOAD=cpu, is far slower and not pursued).
T13 (PR #144 launch, f77c72f6, x8, boost off): lab C1 54.5–55.6, lab prefill 2,433–2,596, kit C1 48.2 (32k: 49.9), kit panel in band (top-1 0.9887, KL 0.00097), agentic whole-run 44.1, min 60-s window 37.9 (floor 50 fails).
Agentic per-request decode is flat across context (median 43.7 / 43.8 / 41.7 tok/s at <8k / 8–20k / 20–40k), so context length is not the limiter; the gap to lab C1 (55) is consistent with expert-cache misses on coding traffic (each miss streams experts over x8 PCIe). Hypothesis, to be tested by changing the cache size.
VRAM budget at ready (server log): expert cache 11.08 GB (auto-fit: free 19.82 − staging 0.48 − reserve 8.26), KV pool 5-bit 1.77 GB for 210k tokens, 3.23 GB still free after CUDA graphs . Context length is therefore a weak lever (the whole KV pool is 1.77 GB); the reserve is the strong one.
Grid (base = PR #144 launch on the latest main-built image a21efea6, GPU2 x8; one variable per step)
Step Change Expected expert cache R1 SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB 8.26 → 6.0~13.3 GB (+20 %) R2 reserve → 5.0 (only if R1 is stable and gains ≥ 3 %) ~14.3 GB (+29 %) N1 SGLANG_EXL3_NGRAM_TIER nvme → pinned (32.6 GB pinned RAM) on the best Rn-gram reads from RAM G1 --max-running-requests 1 --cuda-graph-max-bs-decode 1 (single-stream agentic) on the best, then re-fit the reservefrees graph memory
Metric per step: minimum rolling 60-s window and median per-request decode in the 10-min agentic run (the floor), with lab C1 and lab prefill checked for regressions. Every step runs lab,ttft,stab; the kit panel is re-run on the winner only (cache size does not change the math). Stop rule: 3 consecutive steps < 3 % on the floor metric. If no step lifts the minimum window to ≥ 50, Phase 1 closes with "not reached at x8" , publishing the best config.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Cache misses are a hypothesis: the floor needs +31.9 % (26.4 → 20.0 ms/token); a 20–29 % larger cache need not deliver that. Expert-cache hit/miss and transfer stats are recorded if the engine logs them, else marked unavailable.
Order: R0 (unchanged PR #144 launch on a21efea6) → R1 reserve 6.0 → G1 single-stream graphs on R1 → reserve refit only if peak VRAM (prefill and max-occupancy) shows room → N1 pinned n-grams with host-RAM headroom recorded. R2 (5.0) is not run blind: measured headroom after R1 would be ~0.97 GB, after R2 ~−0.03 GB.
max-running-requests 1 is a recipe concurrency constraint, stated as such.
Floor = minimum rolling 60-s generation-only window after the first 60 s (harness definition); window coverage reported. One 10-min run screens; the winner then needs full §5 qualification, kit band, and ≥ 128k occupied for "agentic-usable". Closure wording: "50 tok/s floor not reached in the tested x8 configurations".
R0 result and re-plan (2026-10-04 22:42 AEST)
R0 (a21efea6, PR #144 launch unchanged): load 260 s, lab C1 55.8, lab prefill 2,419, kit C1 47.4 (32k 48.6), TTFT 2,009 / 2,681, panel in band (top-1 0.9897, KL 0.00097) — then the 10-min agentic run crashed: CUDA OOM . After serving starts, Triton kernels load lazily (log: "device-loaded after serving started … free 0.83 → 0.09 GiB") and consume the 3.23 GB that looked free at ready. So the 8.26 GB reserve is needed on this image; shrinking it cannot grow the expert cache safely. (f77c72f6 survived the same agentic run at the same reserve.) Re-plan: R1 (6.0) is already running and is kept as evidence; R1+G1 dropped . Next: R0 repeat (is the OOM reproducible?), G1 at reserve 8.26 (single-stream frees graph and request memory → headroom), N1 pinned n-grams.
R1 (reserve 6.0, 22:42): load OK, then CUDA OOM at the first lab request — reserve cannot go below 8.26.
R0 repeat, agentic run only (22:58–23:22): no OOM ; 71 tool calls, 0 parse failures, whole-run 43.3, min window 39.7; lab after 57.4 / 2,537. The first R0's OOM is intermittent (it followed lab, kit sweep, TTFT and panel on the same server). Reported as an a21efea6 stability risk for PR #144; frequency unknown from 2 runs.
G1 single-stream at reserve 8.26 (23:22): min window 40.1 (+1 % vs R0 repeat), whole-run 43.1, lab 54.6–55.6.
N1 pinned n-grams (23:48): min window 40.9 (+3.1 %), whole-run 44.6, lab C1 53.7 / 51.3 (lower), prefill unchanged.
Closure (2026-10-05 00:11 AEST)
Stop rule met: R1 no gain (OOM), G1 +1 %, N1 +3 % (within run-to-run spread: two R0 runs and T13 span 37.9–39.7). Phase 1 result: the 50 tok/s floor was not reached in the tested x8 configurations. Best measured agentic floor ~40–41 tok/s (−18 to −20 %); lab C1 55–57, lab prefill ~2,400–2,600, kit panel in band. Best config for publication: the PR #144 launch (image a21efea6, or f77c72f6 which had no OOM in its run). No T16 finalist; T18 registry recipe not prepared; the PR #144 rerun evidence and the kit submission (with this tuning table) are prepared locally for Phase 3.
Independent review (0011-flashnext-x8-tuning-grid.review.md)
AMEND — growing the cache is a reasonable experiment, but the evidence does not establish cache misses as the bottleneck. Flat decode across context only weakens the context hypothesis. The floor needs +31.9% (37.9 → 50 tok/s); a 20–29% larger cache does not imply that speedup. The strongest counter-argument is that coding traffic accesses a broad expert set, so extra capacity may barely reduce misses.
Establish R0 on the new image first. T13 used f77c72f6; the grid uses a21efea6. Comparing directly changes both engine and reserve. Run the unchanged launch on the new, full digest-pinned image, then compare every tuning step against a baseline on that image.
R2 appears to exceed the measured memory budget. Assuming unchanged allocations and consistent units, R1 consumes another 2.26 GB , leaving approximately 0.97 GB after graphs; R2 consumes 3.26 GB , leaving approximately −0.03 GB . “Stable R1” cannot establish that R2 fits. Record peak VRAM during prefill and the claimed occupied-context run before reducing the reserve further.
Concrete alternative: R0 → guarded R1 → G1 → reserve refit → N1. Move graph reduction ahead of R2 when memory is tight, and measure each change separately. Treat max-running-requests=1 as an explicit recipe concurrency constraint; verify that the representative agentic workload fits it. For pinned n-grams, record host-memory headroom and end-to-end performance.
Test the mechanism alongside speed. Record expert-cache hit/miss counts, transferred bytes and transfer stalls if exposed; otherwise mark them unavailable and keep the attribution hypothetical. Going from 37.9 to 50 requires reducing generation time from 26.4 to 20.0 ms/token —a 6.4 ms/token saving. Establish whether expert transfers plausibly account for that much time.
Specify the floor calculation exactly. Ignore the first 60 seconds and calculate rolling windows using generation time only, excluding prefill, tools and idle time. Report window coverage and generation duration. Median request speed and whole-run average are diagnostics; neither establishes the floor.
Separate screening from acceptance. One 10-minute run can select candidates. The winner still needs the three §5 qualification runs—including fully cold and claimed maximum occupied context—≥100 tool calls across its runs , zero parse failures, repetition checks, all six lab gates afterward, and the kit quality band. “Agentic-usable” additionally requires ≥128k occupied tokens .
Narrow the closure claim. Close under the phase timebox or the specified three consecutive measured sub-3% gains, recording skipped branches explicitly. Publish “50 tok/s floor not reached in the tested x8 configurations” rather than implying x8 is exhausted. Prepare results and submissions locally; public publication remains Phase 3.
Runs & Data › Decisions
0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26) decisions/0012-glm-outcome.md
0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)
Result: no GLM-5.3-Flash configuration on this box reaches the registry's 15 tok/s gate; every route is a documented experiment, and no GLM registry PR is prepared. All numbers: lab C1 decode / lab prefill, current layout.
Route Cards Decode Prefill Tools Classification llama.cpp b11312 IQ2_M, ub4096 1 9.6 288 pass, 0 parse failures experiment (1 card < 15) llama.cpp IQ2_M, ub2048, GPU2-first 3 11.8 218 pass experiment (3-card, < 15) llama.cpp IQ3_XXS / IQ4_XS 3 10.4 / 8.3 132 / 156 pass rejected for speed, quality unmeasured llama.cpp Q2_K / IQ3_XXS 1 9.2 / 8.3 78 / 65 pass rejected for speed ik_llama 5bf8f0fe IQ2_M (no MTP) 3 13.4–14.0 185 fails (tool calls unparsed, 0005)experiment-only, unattested image ik_llama IQ2_M + MTP 1 / 3 8.2–12.6 ~150–166 fails experiment-only; OOM / server crash 0xSero glm53-flash-offload, exact mode (x8) 1 7.4–7.5 720–739 pass, 0 parse failures quality reference; route below gate at x8
T25a: no current-layout finalist (decision 0007; rolling-floor failures measured in every 10-min run).
T24: the uncapped candidate-vs-reference comparison runs only for finalists, so it is not run. Exact mode loads on this box (~120 GB host RAM, 23.3 GB VRAM) and stays available as the reference if a future config qualifies.
T25b: 1-card routes < 15 → experiment; 3-card routes < 15 → no separate recipe; ik_llama → experiment-only until a tool-parser fix and an attested local-ai-images build (operator to raise with 0xSero).
T26: not applicable.
Hardware context for the reader: dual-channel DDR4-3200 (~16.6 GB/s host memcpy measured), x8 PCIe (13.5 GB/s), 15 cores at base clock (decisions 0004, 0010). Fast mode (0xSero's 28.9 tok/s route) needs ~238 GB RAM: not possible on AM4.
Runs & Data › Decisions
0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6) decisions/0013-flashnext-oom-fix.md
0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)
Operator request (2026-10-05): rerun the Qwen3.8 (Flash-Next) test and fix the stability issues.
Diagnosis (R0 first run, 2026-10-04 12:32 UTC)
The server died in a prefill: offload_runtime._prefill_parts_compact → `torch.AcceleratorError: CUDA error: unknown error, 29 s after Triton kernel '_row_logsumexp_kernel' device-loaded after serving started (free device mem: 0.00 GiB)`. Earlier in the same lifetime, during the 166k-token context gate, PyTorch logged repeated expandable_segments: memory mapping failed with OOM (free 10 MB of 25.3 GB).
At ready the engine reports 3.23 GB free, but kernels that load lazily after serving starts, plus prefill activations for 12,288-token chunks, consume it. Whether a run crashes depends on which kernels load late and the size of the next prefill: 1 of 3 lifetimes of the unchanged launch crashed; reserve 6.0 crashed at once.
Plan (GPU2 x8, full flow lab → kit sweep → TTFT → panel → 10-min agentic, the sequence that crashed)
Every flow now samples VRAM every 0.5 s (logs/vram/*.csv, summary = minimum free MiB): headroom is measured directly instead of waiting for random crashes.
Runs: baseline (unchanged launch) → A reserve 8.26 → 9.26 GB → B --chunked-prefill-size 12288 → 8192.
Pick the fix with the larger minimum free VRAM and the smaller speed cost (lab C1, kit C1, prefill, agentic floor), then repeat it twice (3 clean lifetimes) before calling it fixed. If neither leaves ≥ 1 GB minimum free, combine them.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Framed as memory-pressure mitigation , not proof the intermittent failure is gone: 3 clean lifetimes would still occur with probability 29.6 % at the old 1-in-3 crash rate (95 % upper bound on crash probability 63 %). The main evidence is the measured VRAM headroom margin; the diagnosis stays a hypothesis (the final unknown error is not itself a proven OOM). Allocator peak/reserved stats are not exposed outside the engine → recorded as unavailable.
Configs, one variable each: baseline (reserve 8.26, chunk 12,288), A (9.26, 12,288), B (8.26, 8,192); (9.26, 8,192) only if neither single change leaves margin. Actual expert-cache GB recorded from the server log for each.
Selection: Flash-Next has no qualifying config at x8 (floor not reached, decision 0011), so the target-based rule cannot pick a winner; the rule is **the config with the largest minimum free VRAM whose lab C1, kit C1, lab prefill and agentic floor stay within 3 % of baseline**; ties → baseline settings. ≥ 1 GB minimum free is an engineering margin, not a spec criterion.
Image provenance: a21efea6 = ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163, local-ai-images run 37113268282 on main, trellis-serve b1e74af; it replaces the PR's branch-built digest per 0xSero's PR #144 comment of 2026-10-03.
Note: the queue skipped the baseline line (identical text already in campaign.done from 2026-10-04); re-added with an explicit step list, so it runs after A and B.
Results (2026-10-05, GPU2 x8, same full sequence lab → kit sweep → TTFT → panel → 10-min agentic)
Config Min free VRAM (0.5 s samples) Lab C1 Kit C1 Lab prefill Agentic floor / avg Outcome baseline (8.26, 12,288) 2 MiB (10:41, kit sweep)49.9 47.8 2,419 — / 43.0 crash in agentic run, 6 tool callsA (9.26, 12,288) 912 MiB 48.1–49.7 43.7 2,408–2,564 37.4 / 41.1 clean, 71 tool calls B (8.26, 8,192) 1,060 MiB 49.5–54.9 48.2 2,131–2,237 39.5 / 43.9 clean, 64 tool calls
Baseline crash: Triton kernels loaded after serving started at "free device mem 0.00 GiB" (alloc_extend_kernel, _row_logsumexp_kernel, _sparse_gqa_chunk_prefill), then CUBLAS_STATUS_INTERNAL_ERROR in cublasGemmEx (cuBLAS could not get workspace). The unchanged launch has now crashed in 2 of 4 lifetimes; it runs the card to ~0 free VRAM. Chosen: B — the most headroom, decode unchanged (kit C1 +1 %, floor −0.5 %), prefill −8 to −12 % (still ~2,100+, over twice the 1,000 target). A costs 8–13 % decode (smaller expert cache). Two more B lifetimes queued.
Confirmation (2026-10-05 10:59–12:29 AEST): fix B, 3 lifetimes
Lifetime Step order Min free VRAM Lab C1 (before / after) Kit C1 Agentic floor / avg, tool calls Outcome B1 lab, kit, ttft, panel, stab 1,060 MiB 54.9 / 49.5 48.2 39.5 / 43.9, 64 clean B2 lab, kit, ttft, panel, stab 1,060 MiB 56.4 / 53.5 47.0 38.8 / 43.2, 62 clean B3 lab, ttft, panel, kit, stab 1,060 MiB 55.0 / 45.0 48.1 40.4 / 44.0, 61 clean
Result: memory-pressure mitigation confirmed for this sequence. The minimum free VRAM is identical in all three lifetimes (1,060 MiB), so the peak is deterministic and the margin is real, against ~0 MiB and a 2-in-4 crash rate for the unchanged launch. 187 tool calls, 0 parse failures across B1–B3. Cost: lab prefill −8 to −12 % (2,131–2,240). Lab C1 varies 45–56 between 30-s windows on every config (single-sample noise), kit C1 is stable at 47–48. Recommendation for PR #144 / the registry launch on 24 GB cards: --chunked-prefill-size 8192. The agentic 50 tok/s floor is a separate, unchanged result (decision 0011): ~39–40 tok/s at x8.
Independent review (0013-flashnext-oom-fix.review.md)
AMEND. The two interventions are reasonable, but the plan can declare “fixed” without demonstrating either the cause or §5 qualification. Treat this as mitigation testing; select the fastest qualifying config, with headroom as supporting evidence.
Strongest counter-argument: three clean lifetimes provide weak evidence of reliability. If the true crash probability remains 1/3, three successes still occur with probability 29.6% . With zero failures in three independent trials, the one-sided 95% upper bound on crash probability is 63.2% . Three runs satisfy the spec’s qualification count only when they meet its prescribed conditions; they do not establish that intermittent failure is eliminated.
The diagnosis should remain a hypothesis. Mapping OOMs and exhausted VRAM strongly support memory pressure, but CUDA error: unknown error does not establish the final failure’s cause. A kernel-loading message 29 seconds earlier is correlation. Record the failing allocation/request and available allocator diagnostics; describe the outcome as “memory-pressure mitigation qualified” unless causal evidence improves.
A 0.5-second VRAM sample is insufficient to certify peak headroom. It can miss allocation bursts, and device-free memory alone cannot distinguish reusable allocator reservations from live allocations. Supplement sampling with per-stage allocator peak allocated/reserved memory, allocation-failure records, and request occupancy/chunk size. Keep collection outside the engine and write logs to tmpfs as §9 requires.
Make the comparisons unambiguous. Use baseline (reserve 8.26, chunk 12,288) , A (9.26, 12,288) , and B (8.26, 8,192) . Explicitly reset reserve for B. Test (9.26, 8,192) afterward if necessary, comparing it against either single-change config. This preserves §10’s one-variable-family rule.
Neither adjustment has a guaranteed memory benefit of the stated size. Raising reserve by 1.00 GB may reduce expert-cache capacity; reducing chunk size by 33.3% reduces only chunk-dependent allocations, not total peak usage. Standardize GB/GiB/MiB, record actual cache capacity, and justify the ≥1 GB threshold against observed peaks. It is an engineering margin, not a spec acceptance criterion.
Replace “larger minimum free VRAM and smaller speed cost” with an explicit selection rule. Require lab decode ≥50 tok/s , lab prefill >1000 tok/s , the quality band, and §5 on the same config at the claimed occupancy. Then choose the fastest qualifying config within the search budget. The cited 53.2 tok/s reference permits only about 6.0% decode loss before missing the target; this machine’s measured baseline determines its actual margin.
Make the three finalist runs conform explicitly to §5: cold with all specified caches reset, claimed maximum occupied context, and repeat; zero OOM/crash/parse failures, ≥100 tool calls across the config’s runs, repetition checks, the rolling decode floor, and all six lab gates after each run. A 166k context gate alone does not demonstrate agentic qualification there.
Resolve provenance before running. a21efea6 is insufficient identification, and it differs from the PR #144 digest specified in Phase 1. Record the full digest, source commit, attestation/allowlist eligibility, and reason for replacing the specified image. Store each immutable launch before execution and publish every failed attempt with explicit unavailable metrics.
Runs & Data › Decisions
0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05) decisions/0014-flashnext-vs-27b.md
0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)
Question: on this box, which is the better local model for agentic coding (and generally): Flash-Next (180B MoE, 6B active, experts streamed from RAM) or the dense 27B the operator runs daily?
Configs (both on GPU2, PCIe 4.0 x8, 15 cores, boost off; one card each)
Flash-Next: PR #144 Phase D launch on a21efea6 with the stability fix (--chunked-prefill-size 8192, decision 0013), EXL3 3.05 bpw, 204,800 context, up to 4 concurrent requests.
27B: the operator's daily server exactly (omarchy-local-ai recipe qwen38-27b-exl3-3bpw-rtx3090-sglang-tp1): ghcr.io/0xsero/sglang-exl3@sha256:84f75f34…, EXL3 3.00 bpw (turboderp/Qwen3.8-27B-exl3@6fe61ad6…), NEXTN/MTP speculative decoding (3 steps, 4 draft tokens), 204,800 context, single stream.
Each is the best known recipe for its model on a 24 GB 3090; they differ in more than one variable by design (this compares models as deployed, not an isolated knob). Sampling, task set, token budget and harness are identical.
Measurements (same order for both; one server lifetime each)
Area Step Metric Speed lab (registry gates), kit sweep, ttftdecode C1 (C2–C4 where supported), prefill 8k–64k, TTFT Long context needle (kit method, thinking off)retrieval at 128k (depths 0.2/0.5/0.8) and ~192k (0.5) Agentic reliability stab (10-min agentic, 8-task §5 set)tool calls, parse failures, loops, decode floor Agentic coding qeval (T07, uncapped, one pass) on tasks-compare30 solve rate (official SWE-bench eval), tokens and wall time per taskFootprint load + VRAM sampler load time, host RAM, VRAM, min free VRAM
tasks-compare30: 30 SWE-bench-Lite test instances drawn with random.Random(20261005).sample (uniform, not hand-picked), images pinned by digest after pull; same 60k completion-token budget per task for both models. With n = 30, a difference in solve rate is reported with a paired comparison (tasks solved by one and not the other) and an exact McNemar p-value; small differences are labelled inconclusive rather than ranked.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Labelled an exploratory deployment comparison on this box (the §10 one-variable rule is waived for it, as the two recipes differ in engine build, MTP and concurrency by design). Conclusions are per priority — agentic coding success, interactive speed, long context, footprint — never "generally better".
compare30 is exploratory: exact McNemar on 30 single attempts needs ≥ 6–0 discordant tasks for p < 0.05 (8–2, a 20-point gap, gives p ≈ 0.11). The site publishes the paired per-task table, the solve-rate difference with a 95 % CI and the p-value; small differences stay unranked.
Cache state: the flow serves cold (container restart, page cache dropped) and then measures in a fixed order, so qeval runs warm for both; the 10-min stab run is screening, not §5 qualification. Flash-Next's kit band is already in band for this config (decision 0013: top-1 0.989, KL 0.00092); the 27B has no kit panel (different model), so its quality evidence is the solve rate only.
Budget: up to 60k completion tokens per task (all of a task's model calls) → worst case 1.8M tokens per model; early submits cut this sharply. Infrastructure failures (image, evaluator) are reported separately from unsolved. Accepted tokens/s and acceptance rate are recorded for the 27B's MTP.
Finding during the 27B run (2026-10-05 18:07 AEST)
The operator's 27B server advertises --context-length 204800, but its KV pool is max_total_num_tokens=135424 (mem-fraction 0.80, fp8 KV, MTP draft buffers). The registry lab's context gate (0.85 × 204,800 = 166,667 tokens) is rejected: "Input length (166667 tokens) exceeds the maximum allowed length (135418 tokens)". So the 27B as deployed really holds ~135k tokens, not 204,800 — the same advertised-vs-real gap seen in the operator's pi session. The comparison reports both; an extra lab run uses the identical server with harness context 131,072 (…-omarchy-ctx131k launch) so its speed gates are measured. Flash-Next's KV pool is 209,984 tokens (fits 204,800).
Independent review (0014-flashnext-vs-27b.review.md)
AMEND. Comparing the best deployed recipes is useful, and differing engines/MTP settings are legitimate for that question. The strongest objection is inferential: 30 tasks, one attempt each, and one server lifetime cannot reliably establish which model is better “generally.” Frame this as a deployment comparison on this machine, with separate conclusions for coding quality, latency, throughput and resource use.
Thirty tasks can miss substantial differences. With exact two-sided McNemar, 5 wins versus 0 losses gives p = 0.0625 ; 6 versus 0 gives p = 0.03125 . Even 8 versus 2—a 20 percentage-point solve-rate advantage—gives p ≈ 0.1094 . Publish the paired outcome table and a confidence interval for the difference. Keep compare30 exploratory; if a definitive ranking matters, predeclare a larger fixed sample before examining results.
One stab run is screening, not §5 qualification. Unless this exact Flash-Next config already has qualifying evidence, require the three qualification runs: cold, claimed maximum occupied context, and repeat; ≥100 aggregate tool calls; the rolling decode-floor measurement; and post-run lab gates. Otherwise label it experiment-only and its stability unqualified. “One server lifetime” needs an explicit exception for the required cold restart.
The measurement order systematically warms Flash-Next’s streamed experts. lab → kit → ttft → needle → stab → qeval measures coding after extensive preparation, whereas cold agent startup may behave differently. Keep the warm deployment comparison, but add the specified cold qualification and cold first-task latency. Disable prefix caching and use fresh prompts for TTFT; record expert, n-gram and OS cache states separately.
Configured context and needle retrieval do not establish agentic usability. A ~192k needle pass supports retrieval at that occupancy, not reliable coding there. Report server-counted occupied tokens and ≥4k generation headroom. Award “agentic-usable” only after §5 qualification at ≥128k occupied tokens; otherwise say “configured for 204,800.”
Restore the explicit Flash-Next quality gate. Require or cite matching evidence for top-1 ≥0.987 and KL ≤0.0013 against the pinned reference panel before any spec headline or registry claim. SWE-bench solve rate complements that requirement; it does not replace it. Publish outside-band results with the prescribed label.
Make the deployment exception reproducible. Explicitly amend §10’s “one variable family” rule for this comparison and avoid attributing differences to architecture alone. Store full image digests, 40-character weight revisions, file hashes, immutable launches, GPU UUID, harness/evaluator commits, ordered task IDs and the source dataset revision. Record accepted tokens/s, MTP acceptance rate and draft settings as §4 requires.
Account for cost and stochastic variation. The completion allowance totals 3.6 million tokens across both models—20 hours of generation at 50 tok/s , before prefill, tools and evaluation. Reserve that time within the phase budget. Define which tokens count toward 60k, distinguish infrastructure failures from model failures, and report total wall time and tokens per successful solve alongside per-task results. A single attempt measures this run’s outcomes; paired statistics do not eliminate sampling variability.
Concrete alternative: run compare30 as a clearly labelled exploratory deployment comparison; reuse documented qualification evidence where it exactly matches, complete missing qualification/cache checks, and publish a tradeoff table. Recommend a model only for a stated priority—such as coding success or interactive latency. Remove “and generally,” and leave small or uncertain quality differences unranked.
Runs & Data › Decisions
0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08) decisions/0015-next-tests-plan.md
0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)
What the thread says
Operator showed local-ai-bench.org/compare and the SWE-bench-Lite contamination finding; asked "Shall I create a benchmark from recent github issues?" — Sero: "Yes" . Operator: "land a novel benchmark, and make it real", and later "an agent/coding benchmark for multiple languages without disclosing the questions/content".
Sero (funding): keep running experiments, turn them into cool demos (Omarchy + the local-ai plugin, "wacky stuff") that he will amplify; fundraiser in a few weeks with 3 goals (small/medium/big) . Hardware advice: stick with NVIDIA (FLOPs, CUDA) — vs the operator's ideas (Epyc 256 GB + 4th 3090, or an M5 Ultra).
Operator wants to go deep on Qwen3.8-Flash-Next ("Qwen 4 is around the corner").
Links: Niko1221/Strata (operator: "amazing decoding numbers … how much better than Qwen 3.8 27B?"); kacper-daftcode/vLLM-Moet + 0xSero/moetier (Sero: "give this to claude your rig will fly", "I don't use gguf"; operator: vLLM-Moet says Blackwell → Sero: moetier "shouldn't need Blackwell"). Operator: "Will test these through the benchmark."
Repo facts: Strata = C++ engine for Flash-Next GGUF quants (Q2_0 … IQ3_S, "Coder" IQ1_M); claims 94 tok/s on a 12 GB RTX 5070 and "100–140 tok/s" expected on a 3090. vLLM-Moet = hand-written SM120 (Blackwell) SASS kernels — cannot run on SM86 3090s. moetier = Sero's data-first standard + simulator for expert placement across VRAM/RAM/NVMe; its loop's primary target is GLM-5.3-Flash on 1× 3090 + 55 GB DDR4 + NVMe at 50 tok/s , with run records (registry/runs) carrying a standard table, utilization and chokepoints (moetier probe).
Plan (ordered; each item ends in published records)
Tell Sero the PR #144 stability finding now (operator sends; draft in prs/pr144-proof.md): a21efea6 runs a 24 GB card to 2 MiB free and crashed 2/4; --chunked-prefill-size 8192 fixes it (3/3, decode unchanged). He is the maintainer, it is the cheapest high-value result we have.
moetier on this box (Sero's explicit ask). (a) hardware record for omarchy-gpu (GPU2 x8 13.5 GB/s, 2-ch DDR4, 16.6 GB/s memcpy, 1 TB NVMe 3.4 GB/s rand) and a Flash-Next model record if missing; (b) moetier probe on the trellis Flash-Next config → utilization + chokepoints run record: name the binding constraint behind the ~40 tok/s agentic floor at x8; (c) reproduce his GLM-5.3 NVMe-mode recipe at a 55 GB RAM budget on our single NVMe (an independent hardware datapoint for his G1/G2 work); (d) moetier sim the fundraiser hardware goals.
Strata vs 27B vs trellis Flash-Next (operator's question). Pinned Strata release; Flash-Next GGUF quants (Q2_0, IQ2_XS, IQ3_S, Coder) on GPU2, then 2-3 cards GPU2-first. Same suite as decision 0014 (lab, kit sweep, TTFT, needle, 10-min agentic) + quality vs the EXL3 3.05 reference (KLD/top-1 where the API exposes logprobs). Test the "100–140 tok/s on a 3090" claim. GGUF → operator-facing comparison, not a registry recipe (Sero).
Fresh-issues benchmark (decontaminated, held out). Pilot: ~50 Python tasks from issue-linked PRs merged after 2026-09-01 (after both models' release), auto-built Docker envs (reuse SWE-rebench / SWE-bench-Live RepoLaunch tooling), validated FAIL_TO_PASS/PASS_TO_PASS with the gold patch, run through our harness. Held out: publish aggregate scores, repo list, task-ID hashes and a few retired samples; refresh monthly. Then add TypeScript, Go, Rust (multi-language, per the operator). Keep the gold-overlap check as a standing guard.
Run the three contenders on it : 27B, trellis Flash-Next (fixed), best Strata quant; paired analysis.
Demos for Sero / the fundraiser : Omarchy + local-ai plugin — (a) 27B as a fast coding agent (one-shot app), (b) Flash-Next answering over a ~190k-token codebase; short screen recordings + numbers from the records.
Fundraiser evidence : per goal (small: 4th 3090 / 2nd NVMe; medium: Epyc 8-ch 256 GB + x16 lanes + 4th 3090; big: a Blackwell card → vLLM-Moet/moetier), projected tok/s from moetier sim calibrated on our measured runs.
Qwen 4 day-0 readiness : launch templates + the fresh-issues set ready so a new model runs the same day.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Milestone 0 first: Phases 1–2 are closed as documented results (0011, 0012); their Phase 3 material (PR #144 evidence, kit submission) goes out before new work is presented as findings. PR #144 wording: "no crashes in 3 full-sequence runs with chunked prefill 8192 (vs 2 of 4 without); decode unchanged within ±2 % (kit C1 47–48); prefill −8 to −12 %" — a mitigation, not a proven fix.
moetier: answer one measurable question — what binds the ~40 tok/s agentic floor (bytes per token over PCIe, expert-cache misses, CPU lane, GPU idle) — with measured counters, then change one variable family. The GLM reproduction states its RAM budget in GiB including pinned memory, swap off, cache resets declared; 50 tok/s is Sero's hypothesis, and registry gates still apply.
Strata: screen quants on GPU2 only (one 10-min agentic run + speed suite each), qualify only finalists, and add 2–3 cards only for a finalist with a measured reason. Verify what "Coder" changes (weights/behaviour). Quality without logprobs = "unavailable", never in-band.
Fresh-issues set = "temporally held out" , not "decontaminated": require issue and fix after 2026-09-01, overlap check against existing benchmarks, no network access for the agent during evaluation, tasks frozen before any tuning. A separate small development set is used for choosing configs; finalists run once on the held-out set. 50 tasks is a pilot (±14 pp at 50 %); report paired, repository-aware uncertainty.
Secrecy vs reproducibility: a private immutable manifest (commits, patches, tests, env digests, harness settings) with a published commitment hash; controlled reproduction on request; full artifacts released when a task set retires. Active-set raw logs stay out of the public site (SPEC §8 amended accordingly).
Fundraiser numbers are conditional projections with stated assumptions and sensitivity ranges, kept apart from measured results; topology (slots, lanes) is checked before any 4th-3090 projection.
Independent review (0015-next-tests-plan.review.md)
AMEND. The strongest objection is scope drift: the agreed deliverable is qualified Flash-Next and single-3090 GLM recipes; this plan adds a benchmark product, several engine comparisons, demos and hardware projections before closing those deliverables. Keep the new work, but put it behind an explicit milestone: publish the fastest qualifying configs—or documented failures within the spec’s timebox—and prepare their submissions.
Report PR #144 as a promising mitigation, not a proven fix. Moving from 2/4 crashes to 0/3 is useful evidence, but three successes still permit a per-run failure probability of 63% at a one-sided 95% confidence bound under independent, identical trials. Use “no crashes in three qualification runs,” verify the same occupied context and workloads, and quantify “decode unchanged.” Confirm Phase 3 publication requirements are already satisfied or explicitly amend them.
Make moetier answer a measurable bottleneck question. Going from ~40 to 50 tok/s requires 25% more throughput , or 20% less time per token. If PCIe alone were saturated at 13.5 GB/s, that implies reducing traffic from approximately 338 to 270 MB/token . These are conditional bounds, not a diagnosis. Measure transferred bytes, expert-cache misses, CPU work and GPU idle time, then change one variable family. A utilization report alone cannot establish causality.
Define the GLM reproduction before running it. Specify whether “55 GB” means decimal GB or GiB, and whether it includes pinned allocations, page cache and engine overhead; declare swap and cache-reset behavior. Treat 50 tok/s as the supplied recipe’s hypothesis, not an expectation. Registry eligibility still requires the ≥15 tok/s floor, uncapped quality evaluation and ≥80% of reference solves; reproducing a fast simulator or engine run does not satisfy those gates.
Separate Strata’s speed claim from its eligibility. The supplied 100–140 tok/s claim is 2–2.8× the Flash-Next target, but needs matching context, cache state, sampling and accepted-token accounting. A low-bit quant can win throughput and fail quality. Missing logprobs mean “quality unavailable,” which cannot establish the required top-1 ≥0.987 / KL ≤0.0013 band. Verify whether “Coder” changes weights or model behavior before presenting it as a quant-only comparison.
Rename the fresh set “temporally held out”; “decontaminated” overclaims. A PR merged after model release can fix an issue published long before release, and an agent with network access can retrieve the public solution. Require fresh issue/test provenance, inspect overlap with existing benchmarks, block solution retrieval during evaluation, and freeze evaluation tasks before tuning. These controls reduce contamination risk; they do not prove its absence.
Treat 50 tasks as a pilot, not a decisive leaderboard. At a 50% solve rate, 50 independent tasks give an approximate 95% margin of ±14 percentage points ; one task changes the score by 2 points. Repository clustering can make uncertainty larger. Report paired outcomes and uncertainty based on discordant tasks, with repository-aware analysis. Use a development set for choosing the “best Strata quant,” then evaluate finalists once on the held-out set.
Resolve secrecy versus reproducibility explicitly. Public aggregate scores, repository names and task-ID hashes do not let another evaluator reproduce scores, and hashes of enumerable public IDs may reveal tasks. Keep a private, immutable manifest containing base commits, patches, tests, environments and harness settings; publish its commitment and arrange controlled independent reproduction. Amend §8 so public raw logs cannot accidentally expose active tasks, with full artifacts released upon retirement.
Make fundraiser projections conditional, with uncertainty. A fourth 3090 adds 24 GB nominal VRAM , not guaranteed usable bandwidth or speed; the actual slot, lane and chipset topology must be established first. A second NVMe helps only if measured storage contention is binding. EPYC and Blackwell projections require calibration beyond this AM4/SM86 machine. Publish assumptions and sensitivity ranges; keep measured results separate from simulated forecasts.
Concrete replacement order: narrow PR #144 report → close existing Flash-Next/GLM qualification and submission material → probe the measured bottleneck → screen Strata on GPU2 and qualify only finalists → freeze and validate the 50-task pilot → paired contender evaluation → demos and conditional fundraiser scenarios. Four quants across three layouts already imply 12 configs: up to eight hours of screening plus qualification alone , before loading, lab, sweep and quality checks. Reserve multi-card expansion for a single-card finalist with a demonstrated reason to benefit; label the ~190k demo by actual occupancy and qualification status.
Runs & Data › Decisions
0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09") decisions/0016-fresh-issues-stage2.md
0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")
Context: decision 0015 item 3 (temporally held-out Python set, issue AND PR created ≥ 2026-09-01, private tasks, published scores + commitment hash). Stage 1 (tools/fresh/, data in ~/dev/local-ai-exp/fresh-private/, never in this repo) gave a pool of 5,450 candidates and a ≤ 3-per-repo shortlist of 763 (275 repos; 249 easy / 271 medium / 243 hard; 528 from repos not in any public SWE dataset). Operator choice (2026-10-08): the setup/filter LLM is this Claude subscription (claude -p, Opus) — not an API key; Anthropic therefore sees which tasks we chose ("semi-private for Anthropic models", stated next to every score).
Plan
Stage-2 input (deterministic, before any LLM): from the shortlist drop rows with base commit dated before 2026-08-24 (5), no .py test file (5), and repos created after 2025-09-01 or with < 10 distinct contributors in the last year (guards against single-author agent-generated issue→PR streams seen in the top repos). Sample 150 by difficulty (easy 45 % / medium 40 % / hard 15 %), seen/unseen repos ≈ 1:2. Seed fixed, list recorded.
Clarity filter (Opus, one claude -p call per task, JSON out): the SWE-bench-Verified rubric — is the issue well specified, can a correct fix be judged by the hidden tests without guessing names/messages that only the tests define, is the fix given away in the issue. Reject scores ≥ 2 (0–3 scale) on either axis.
Environment (deterministic skeleton + agent-written setup):
Repo snapshot: git init + git fetch --no-tags origin <base_commit> (ancestors only; no branches, tags or later objects exist in the image), checkout detached, remote removed, reflog expired, gc --prune=now; a post-build check asserts git rev-list --all contains no commit dated after the base and the gold blobs are absent.
Base image python:<ver>-bookworm pinned by digest. Opus (claude -p --model opus, Bash tool restricted to docker exec into a throwaway per-task container with network, 45-min cap, ≤ 60 turns) writes setup.sh (system packages, pip/uv installs) and the test command pytest -rA <test files>; installs pinned to versions available before the PR's creation date where the agent can (uv pip --exclude-newer). The final image is rebuilt from setup.sh without the agent (reproducible), tagged lab/fresh.<id>, digest recorded.
LLM-written log parsers are not used: pytest -rA + swebench parse_log_pytest only (no host-side code from the agent).
Validation (no LLM): in the image, run the test files on base + test_patch (expect failures), then base + test_patch + gold patch; FAIL_TO_PASS = fail→pass, PASS_TO_PASS = pass→pass. Require ≥ 1 F2P, gold resolved 3/3 (flaky tests dropped from P2P, task dropped if an F2P is flaky), empty patch resolved 0/1, test run < 10 min. Rows emitted in our swebench 5.x format (eval_script, parse_log_pytest, version fresh-2026-09, image digest) into a private task dir; swebench evaluates them with network off for the agent under test.
Freeze: from validated tasks, re-apply ≤ 3 per repo, stratify, pick 50 test + 10 dev (dev is for harness debugging only; never reported). Manifest (ids, base, patches, digests) committed only in fresh-private; publish sha256(manifest || salt) and the counts/distribution now, the salt and manifest when the slice is retired.
Runs: Qwen3.8-27B (ctx131k), Flash-Next trellis c8192, Strata finalist; same harness as compare30 (uncapped, one pass), paired analysis (McNemar), gold-overlap per task as a leak check (flag > 50 % verbatim reproduction).
Scheduling and cost
Stage 2 is CPU/disk heavy (docker builds, test runs): it runs only as sh lines of the campaign queue, never during a GPU measurement. Parallelism 3 tasks. Disk: ~1–2 GB per image → ≤ 200 GB worst case before pruning failed builds. Subscription usage: ~150 filter calls + ≤ 150 setup sessions; stop and report if rate-limited.
Risks
Yield may land < 50 (published Python validation yield ~19 %): widen to the pool (reserves) and to October issues as a second slice rather than relaxing the date rule. A repo snapshot without later commits can still contain the fix if it was committed to the base before the PR (rare; the gold-overlap check at run time flags it).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Supplementary pilot , contamination-reduced (not "uncontaminated"); labels next to every score: "Opus-curated", "semi-private for Anthropic models", rubric + Opus id + rejection counts published. Does not replace §5 qualification or the reference panels. Public release only after Phase 3 review.
Ordered batches, not 150: an ordered September list (stage-2 filters applied, stratified, seed fixed) is frozen now; attempts run in batches of 50 until 60 validated (expect ~320–380 attempts). Budgets: setup cap 30 min/task, ≤ 3 parallel, stage-2 wall time ≤ 40 h, image disk ≤ 150 GB (rejected images pruned immediately). If a budget runs out, stop with an undersized pilot and say so. October = separate slice.
Git invariant: image HEAD == base SHA; refs = HEAD only; git rev-list --all == ancestry of base; no stray working-tree files vs git status; gold-patch files are never mounted or copied into setup containers. Base must fail the hidden F2P tests (acceptance check).
Reproducibility: keep setup.sh, resolved pip freeze/uv lock with hashes, apt package versions, build logs, test command and image digest. Setup containers: CPU/RAM/PID limits, no host secrets, no Docker socket.
Validation: collected node IDs must match between runs; skips/xfails/collection errors counted explicitly; test patch must not disable tests; each repetition in a fresh container; gold 3/3 + empty 0 recorded with every exclusion and reason (3/3 is weak evidence on flakiness — reported as such).
Protocol frozen before candidate runs: manifest bytes, high-entropy salt, ordered reserves, selection rules, harness/parser SHAs, sampling, per-task token budget (same for all models; the compare30 harness settings), tool budget, context policy, retries, timeout treatment. Weight revisions recorded per model.
Reporting: one primary comparison predeclared — Strata finalist vs Qwen3.8-27B (the operator's question); others exploratory. Discordant counts, exact McNemar, repo-clustered intervals, ±14 pp margin stated at n = 50.
Gold overlap: descriptive audit only (denominator = gold added lines), never used to exclude or rescore.
Predeclared before any candidate run (2026-10-09 05:50 AEST)
Strata finalist = IQ3_S (launch strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb): T41 screen agentic floor 87.3 tok/s (Q2_0 123.3), compare30 29/30 (Q2_0 28/30); chosen as the higher-quality quant that still clears the 50 tok/s floor. Primary comparison: Strata IQ3_S vs Qwen3.8-27B (ctx131k launch) , exact McNemar on the 50-task test slice. Exploratory: Flash-Next trellis c8192, Strata Q2_0.
Harness: tools/agentic/quality_eval.py at the commit recorded in each result, uncapped one pass, 60,000 completion tokens per task, sampling temperature 0.7 / top_p 0.95 / seed 1234 / max_tokens 8192 (compare30 settings), agent container network none.
Runs & Data › Decisions
0017 — Site structure: story-led home, findings, recipes, model cards decisions/0017-site-structure.md
0017 — Site structure: story-led home, findings, recipes, model cards
Operator request (2026-10-10): pages were messy; home must show the sequence of what was done and a summary of achievements; every model gets a model card (link, details, learnings to run). Debated with GPT-6 Astra (Codex, 2 rounds) before building.
Outcome
Navigation: Home · Findings (speed, coding quality, GLM-5.3-Flash, hardware) · Recipes · Models · Method & Hardware · Runs & Data · Funding & Contributions. Old tabs (#charts, #compare, #fresh, leaderboards) kept for links, marked superseded or moved under Data.
Home: dated current answer (speed config and task-success config named separately), three bounded achievement summaries with conditions/limits/evidence, the 10-step dated timeline (outcome-led, chips, decision links), contributions with real status, hardware hypotheses, next steps.
Editorial text lives in site/content.json (operator-reviewed); charts and numbers in tables are computed from records.
Proposal (Claude)
local-ai-bench.org — information architecture proposal (draft by Claude)
Context
Static site generated from ~250 result records (tools/site/build.py). Audience, in order: (1) 0xSero, maintainer of the local-ai registry/recipe-kit and our lead, (2) potential fundraiser backers (hardware goals small/medium/big), (3) local-LLM practitioners who want a recipe for 3090-class boxes. The operator says the current pages "look very messy between them".
Current tabs (10 + 247 run pages)
Overview (per-model status cards, PR list, recent runs, a new headline callout) · Flash-Next leaderboard · GLM leaderboard · 27B leaderboard · Fresh-issues benchmark (method + results mixed) · Flash-Next vs 27B (public SWE-bench-Lite comparison, memorised) · Charts (8 charts with no narrative) · Runs (247) · Hardware & methodology (hardware, rules, decisions, spec). Problems: the story is nowhere; results for one question are spread over 4 tabs; three near-identical leaderboards; charts detached from the claims they support; method and results mixed; the old compare page contradicts the new benchmark unless read with a caveat.
What was done (chronological, all dated AEST, each backed by records + a decision file)
Oct 3–4 · Box baseline: 3x RTX 3090 on AM4 (GPU2 x8 13.5 GB/s, GPU0 behind chipset 6 GB/s), faulty CPU core found and quarantined, boost off after machine-check errors (decisions 0004, 0010). Card order matters: GPU2-first +43–51 % prefill (0006).
Oct 4 · Flash-Next on trellis/SGLang (Sero's PR #144 launch): lab 55 tok/s, prefill 2,400, but agentic floor ~40 at x8 — 50 not reached in any tested x8 config (0011). x16 card move dropped by the operator (0009).
Oct 4–5 · GLM-5.3-Flash on one 3090: every route < 15 tok/s (0012).
Oct 5 · PR #144 image OOM found (2/4 crashes) and mitigated with chunked prefill 8192 (0013). Public site launched.
Oct 5–6 · Flash-Next vs 27B on public SWE-bench-Lite: 30/30 vs 18/30 — but Flash-Next reproduces upstream fixes verbatim (memorised) (0014).
Oct 8 · From the Sero thread (0015): PR #144 rerun comment + kit submission PR posted; Strata (C++ GGUF engine) screened: Q2_0 floor 123 tok/s, IQ3_S 87 tok/s on one x8 card; moetier on the box: GLM NVMe recipe 6.1 tok/s at 55 GiB → 11.3 at 90 GiB; probe shows NVMe-bound → CPU-bound; trellis Flash-Next is PCIe-bound.
Oct 8–10 · Built a private fresh-issues benchmark (0016): 430 attempts → 50 tasks, issue+PR after 1 Sep 2026, commitment hash published. Strata IQ3_S 48/50 vs 27B 36/50 (exact McNemar p = 0.0018, predeclared).
Proposed structure (5 top-level sections)
Home = the story. Hero: the current answer in one sentence + 3–4 achievement tiles (fastest agentic setup on one 3090; best coding quality on fresh tasks; GLM verdict; contributions upstream). Then the dated timeline above as a vertical sequence, each step with its one key number, a status chip (result / negative result / fix / tool) and a link to the finding. Ends with "What's next" (Qwen 4 readiness, multi-language slice, fundraiser hardware sims).
Findings (one page per question, narrative + the charts that support it + links to runs): a. Flash-Next speed on one RTX 3090 (engines compared; why x8 PCIe caps trellis; Strata result; stability fix) b. Coding quality (fresh-issues results; the memorisation finding on public tasks; 27B vs Flash-Next) c. GLM-5.3 on one RTX 3090 (routes tried; moetier NVMe recipe; what limits it; what hardware would fix it) d. Hardware lessons (PCIe lanes and card order, faulty core, NVMe at x2, RAM caps)
Methods : how we measure (lab gates, kit sweep, TTFT, stability floor, quality panel, probe) and the fresh-issues benchmark construction (moved out of the results page), hardware record, rules for statuses.
Data : one filterable config table (model/engine/quant/cards filters, replaces the three leaderboards), all runs, run detail pages, decisions log.
Upstream : PRs and submissions to Sero's repos with status, and recipes ready to copy (launch commands).
Charts tab removed: every chart moves into the finding it supports. The old "Flash-Next vs 27B" page becomes a section of Findings b, framed as the memorisation evidence.
Questions for the reviewer
Is this the right structure for these audiences? What would you cut, merge or reorder? Is a timeline the right home page, or should the home lead with findings? Anything that makes the site look less credible?
Round 1 (GPT-6 Astra)
The proposal fixes the fragmentation, but risks replacing ten messy tabs with a polished lab diary. Lead with findings and usable recipes; make the chronology secondary. A public visitor needs to know what works, under which conditions, and how strong the evidence is.
0xSero’s priorities deserve a prominent contribution summary, not control of the entire navigation. Practitioners need reproducible setups; backers need evidence that a specific purchase would resolve a measured constraint. Neither audience benefits from reading seven dated steps first.
I would use five sections, in this order:
Results — question-led findings, with a comparison summary.
Recipes — tested launch commands, requirements, expected performance, limitations.
Method & Hardware — measurement definitions, benchmark construction, exact machine configurations.
Runs & Data — filterable results, individual runs, downloadable records, decisions.
Project & Funding — upstream contributions, dated progress log, hardware proposals.
Home remains accessible through the logo. “Upstream” is insider language and combines two different jobs: tracking contributions and getting software running. Recipes should not be buried there.
The home page should contain, in order:
A bounded answer: which tested configuration you currently recommend for which workload, on what hardware, as of what date.
Three finding cards: practical agent performance, fresh-task coding results, and GLM feasibility. Each needs a configuration, metric, limitation, and evidence link.
“Run a tested setup” linking directly to recipes.
“What additional hardware would test” showing small/medium/big funding options, the measured constraint each addresses, cost, and success criterion.
“Recent changes” with three entries and a link to the full timeline.
Cut the achievement tiles. “Fastest agentic setup on one 3090” is especially risky when the supporting numbers mix laboratory throughput, agentic floors, engines, and quantizations. “Best coding quality” overstates a two-configuration, 50-task comparison. Use “Highest measured [defined metric] among tested configurations” and “48/50 versus 36/50 on this task set.” Put contributions below results, not alongside them as equivalent scientific achievements.
Keep the question-led findings, but sharpen their titles:
Which setup runs coding agents fastest?
How often do the tested setups solve fresh issues?
Is GLM practical on this machine?
Which hardware constraints affect performance?
The largest credibility issue is comparability , not navigation. “One RTX 3090” obscures a three-card host with unusual lane allocation, CPU restrictions, RAM requirements, and NVMe constraints. Display active GPUs separately from host hardware. Every comparison should expose model, engine, quantization, workload, context length, and relevant resource settings.
The fresh benchmark needs a compact evidence box beside its headline—not only a distant Methods link. Explain how 430 attempts yielded 50 tasks, exclusions and their timing, whether selection involved the evaluated systems, paired outcomes, and what was predeclared. A commitment hash needs a timestamp and a clear account of what it commits to. Recent issue dates alone do not establish contamination freedom; the p-value does not establish general coding superiority.
Keep chart integration, consolidated data, negative results, and the decisions log. Preserve old URLs with conspicuous superseded-result notices. Describe verbatim fixes as evidence consistent with memorisation unless stronger provenance establishes it. Treat proposed hardware remedies as hypotheses until tested.
Round 2 (GPT-6 Astra, after the owner's constraint)
The owner’s constraint changes my recommendation: make chronology the home page’s main structure, with the current answer and achievement summary above it. I withdraw the blanket objection to tiles; bounded result summaries serve a useful purpose.
1. Home, top to bottom
Current answer. A short, dated statement distinguishing the fastest tested configuration from the configuration evaluated for task success. Explicitly separate Q2_0’s speed result from IQ3_S’s 48/50 result. Name the workload, active GPU count, and three-card host; link to the applicable recipes.
Three achievement summaries: agentic performance, fresh-task outcomes, and GLM feasibility. Each gets a metric, essential conditions, limitation, and evidence link. Keep them compact enough that the answer and summaries fit a typical desktop first screen; on mobile, prioritize the answer. Put the fresh-benchmark evidence box directly beside or beneath its summary, not behind a tooltip.
“How we reached this answer.” The dated, chronological sequence is the page’s main body: 6–8 outcome-led steps, each linking to its finding and underlying records. Show the initial constraint, subsequent result, and any change in interpretation. Include the public-benchmark result together with the memorisation caveat.
Contribution summary. Show the stability work, rerun report, and kit submission with accurate statuses and links.
“Run a tested setup.” Direct recipe links with requirements and expected results.
“What additional hardware would test.” Measured constraints, proposed purchases, costs, hypotheses, and success criteria.
2. Final top-level navigation
Home → Findings → Recipes → Method & Hardware → Runs & Data → Funding & Contributions
Home deserves an explicit label because the owner makes the sequence a primary destination. Combining funding and contributions keeps project support together without burying recipes.
3. Remaining critical objections
Your timeline revision works, but “one number per step” should be a preference, not a rule . Some outcomes require a denominator, before/after values, or paired conditions. Clarity matters more than visual uniformity.
The proposed tiles still need corrections:
“Strata Q2_0” identifies an engine and quantization, not the model. Include the model and define “floor”; ten minutes alone does not explain that metric.
“GLM-5.3” must preserve the exact tested variant, apparently GLM-5.3-Flash , and say “across tested routes on this host.”
“Predeclared” must identify what was declared and when. The p-value belongs with paired outcomes and selection disclosures.
“2 upstream contributions” must distinguish submitted, accepted, and merged work.
Most critically, never combine Q2_0 speed and IQ3_S task success into an implied single configuration’s achievement.
Runs & Data › Decisions
T41 — Strata (Flash-Next GGUF) screen on GPU2 (decision 0015 item 3) tickets/T41-strata-screen.md
T41 — Strata (Flash-Next GGUF) screen on GPU2 (decision 0015 item 3)
Pinned Strata v0.1.40.3 (d5ea713, local sm86 build), ISTA-DASLab GSQ-RCO GGUF quants at ed59f920, one RTX 3090 (GPU2, x8). Screen each quant with lab,kit,ttft,stab (same suite as Flash-Next trellis and 27B). No KL panel: Strata's CUDA server has no logprobs → quality = "unavailable via API"; quality comes from the coding evals instead. Order: Q2_0 → IQ3_S → Coder IQ1_M (pruned 256/512 experts) → IQ2_XS if useful. Finalists (best speed with tool calls clean) then run needle (ctx permitting) and the fresh-issues eval. Strata serves one request at a time: kit C2–C4 cells measure queueing, not batching. 2–3 cards only with a reason (e.g. to lift the 131k context). Claim to check: "100–140 tok/s on a 3090" (author estimate). Smoke 2026-10-08: Q2_0 130.5 tok/s on a 532-token code answer.
Runs & Data › Decisions
T42 — moetier on omarchy-gpu (decision 0015 item 2) tickets/T42-moetier.md
T42 — moetier on omarchy-gpu (decision 0015 item 2)
Sero's ask: run github.com/0xSero/moetier (f751040) here. Hardware record: moetier/hardware/omarchy-gpu-rtx3090x8-kc3000.json.
GLM-5.3 NVMe recipe (glm53-rtx3090-55g-nvx4 adapted; image sybil-solutions/glm53-flash-offload v4.1-nvme 99b8926e): 55 GiB (his budget, sim ~7 tok/s here: single x2 NVMe) and 90 GiB (sim ~15) RAM caps. Steps lab,kit,panel,ttft with probe record over the whole server lifetime; panel = his GLM exl3 reference panel.
Flash-Next trellis c8192 with probe: utilization behind the ~40 tok/s agentic floor (chokepoints need his instrumented build, not public → utilization only).
Sims for fundraiser goals: research/moetier-onboarding.md §3.
Store: ~/models/local-ai-exp/nvx-glm53-v4.1 (117.3 GB, built 88 s, verify 2.48 GB/s).
Results (2026-10-08, GPU2 x8, boost off; probe = tools/probe_summary.py over moetier probe samples)
Config Lab C1 Lab prefill Kit decode C1/C2/C3/C4 · C1@32k Kit prefill 8k–64k Panel Binding (probe) GLM v4.1 NVMe, 55 GiB (his budget) 6.1 314 5.2 / 4.85 / 4.71 / — · — (2-h step cap) 181–340 top-1 1.0, KL 0 NVMe : 3.05 GB/s mean, ≥ 80 % of 3.36 ceiling 88 % of the kit; PCIe rx 3.3 GB/s; CPU lane 61–75 %GLM v4.1 NVMe, 90 GiB 11.3 630 10.95 / 10.99 / 10.9 / 10.53 · 10.48 379–676 in band CPU lane / host threads : lane 80–83 %, main+ctl spinning ~90 %; NVMe 1.3 GB/s (not saturated); PCIe rx 6.4 GB/sFlash-Next trellis c8192 (probe arm) 55.3–56.1 2,107–2,222 48.5 / 53.5 / 55.8 / 53.9 · 49.1 2,279–2,435 (not run) PCIe x8 : rx mean 7.8–10 GB/s, p95 14.3 (= ceiling), ≥ 80 % of ceiling 20–43 % of the time; CPU 4 %, NVMe 0; agentic floor 39.0
Sim vs measured (moetier sim on trace G002): 55 GiB 7.0 → 6.1 (C1 lab) / 5.2 (kit); 90 GiB 15.0 → 11.3 / 10.95. The sim's CPU-lane cost is EPYC-calibrated (22 threads, 8-ch DDR4); ours is 13 lane threads at 3.4 GHz on 2-ch DDR4, so the sim is optimistic once the NVMe tier stops binding. Chokepoint attribution beyond utilization needs the engine live counters (his instrumented build; not public).
Runs & Data › Decisions
T43 — Runs on the private fresh-2026-09 slice (decision 0016) tickets/T43-fresh-slice-runs.md
T43 — Runs on the private fresh-2026-09 slice (decision 0016)
50-task test slice (issue + PR created ≥ 2026-09-01, Opus-curated, validated gold 3/3 / empty 0, ≤ 3 per repo), frozen with a published commitment hash. Configs (GPU2): Strata IQ3_S (finalist), Qwen3.8-27B ctx131k, Strata Q2_0, Flash-Next trellis c8192. Primary comparison predeclared in 0016: Strata IQ3_S vs 27B (exact McNemar); others exploratory. Labels on every score: contamination-reduced, Opus-curated, semi-private for Anthropic models.
Freeze (2026-10-09 11:01 AEST)
Commitment: sha256(manifest bytes || 32-byte salt) = 5371c3b8d1caedc134657a17b86cad94fc7fe13d81365e932d068f5e22a9c710 (manifest + salt private; revealed when the slice is retired).
Test 50 (24 easy / 20 medium / 6 hard; 47 repos; 15 from repos in public SWE data), dev 10 (never reported).
Funnel: 430 attempts → clarity rejects 327 (tests misaligned 161, fix given away 95, underspecified 71) → 103 to setup/validation → 62 validated (setup/validation rejects: base eval error 7, test patch disables tests 7, no F2P 5, setup timeout 3, base test timeout 2, flaky F2P 1, gold eval error 1, disallowed pytest arg 1, agent gave up 1). Stage-2 wall 6.3 h, images 46.9 GB.
Strata IQ3_S (2026-10-09, 10 h, 83.0 tok/s)
48/50 solved (96 %); 3,366 tool calls, 1 parse failure, 1 truncation, 6 repetition hits. Leak checks: every test image has a clean tree at the base commit (git status empty); F2P test functions found in 12 images are pre-existing tests the hidden test patch modifies (normal). Gold overlap (share of upstream added lines reproduced verbatim): median 0.20 , 3/50 ≥ 0.90 (vs median 1.00 on SWE-bench-Lite compare30) → not memorized. Caveat: the slice skews easy (median gold fix 7 added lines; 6/50 hard) — ceiling effect possible; 27B decides whether it discriminates.
Primary comparison and exploratory (2026-10-10)
Config Solved (50) easy / medium / hard Decode tok/s Tool calls Parse fail Truncations Repetition hits Strata IQ3_S 48 23/24 · 19/20 · 6/6 83.0 3,366 1 1 6 Strata Q2_0 46 — 123.1 2,748 0 3 — Qwen3.8-27B ctx131k 36 19/24 · 14/20 · 3/6 118.6 2,682 4 39 67 Flash-Next trellis c8192 running
Primary (predeclared): Strata IQ3_S vs 27B — discordant 13 vs 1, exact McNemar p = 0.0018. Survives Bonferroni for 3 comparisons. Exploratory: Q2_0 vs 27B 10 vs 0 (p = 0.002); IQ3_S vs Q2_0 3 vs 1 (p = 0.63, no difference shown). Repo clustering: 47 repos for 50 tasks, so task-level ≈ repo-level here. Caveat on 27B: 39 output truncations and 67 repetition hits (vs 1 / 6) — part of its gap is looping/runaway generation under these sampling settings (temperature 0.7, max_tokens 8192), which is a property of the configuration as run. Labels: contamination-reduced, Opus-curated, semi-private for Anthropic models.
Flash-Next trellis c8192 on the slice: stopped by the operator at 9/50 (2026-10-10 13:20 AEST) — same weights as Strata at ~43 tok/s, not expected to differ measurably at n = 50; recorded as not completed (no result record).
Runs & Data › Decisions
T44 — Resource profiles: RAM, CPU, VRAM per phase (operator request 2026-10-10) tickets/T44-resource-profiles.md
T44 — Resource profiles: RAM, CPU, VRAM per phase (operator request 2026-10-10)
What each setup uses while it works, on GPU2 (x8): idle (loaded), prefill (3 fresh ~32k-token prompts, 1 output token), decode (3 prompts, up to 2,048 output tokens). tools/measure/resources.py: moetier probe at 0.1 s (VRAM, GPU util, PCIe, per-CPU busy, container memory) + container memory.stat (anon vs page cache) at each phase end. Configs: Strata Q2_0, Strata IQ3_S, Qwen3.8-27B ctx131k, Flash-Next trellis c8192.
Runs & Data › Decisions
T45 — Strata Coder (Qwen3.8-Flash-Next Coder, GSQ-RCO IQ1_M): full suite (operator request 2026-10-11) tickets/T45-strata-coder.md
T45 — Strata Coder (Qwen3.8-Flash-Next Coder, GSQ-RCO IQ1_M): full suite (operator request 2026-10-11)
ISTA-DASLab's expert-pruned Flash-Next (256 of 512 experts kept, ~3.5 bpw, 58.4 GB), on Strata v0.1.40.3, GPU2 x8. Steps: lab, TTFT, 10-min agentic stability, resource profile (idle/prefill/decode), needle (98k), SWE-bench-Lite compare30 (memorisation check), fresh-2026-09 test slice (exploratory vs the predeclared primary). Not applicable: kit sweep (SGLang-native API), reference panel (no logprobs).
Runs & Data › Decisions
T46 — OrcaRouter Qwen3.8-Flash-Next-Uncensored IQ3_XXS on Strata (operator request 2026-10-11) tickets/T46-orcarouter-uncensored.md
T46 — OrcaRouter Qwen3.8-Flash-Next-Uncensored IQ3_XXS on Strata (operator request 2026-10-11)
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF@e43d00f4 IQ3_XXS (85.2 GB, 2 shards), Strata v0.1.40.3 via the documented manual compatibility pack (docs/ORCA.md, --compat-bf16), original MTP draft head, GPU2 x8, 131k context like the other Strata configs. Steps: lab, TTFT, 10-min agentic stability, resource profile, our fresh-issues benchmark (fresh-2026-09 test slice, exploratory vs the predeclared primary), then needle (98k) and SWE-bench-Lite compare30 (memorisation check). Not applicable: kit sweep (SGLang API), reference panel (no logprobs).
Results (2026-10-11/12, GPU2 x8)
Speed: lab C1 73.6–76.4, prefill 2,105–2,122, TTFT 8k/32k 2,007/2,162, agentic floor 86.0 (run marked failed: 2 repetition hits; 0/2 tasks in 10 min), resource profile decode 92.8 tok/s, 23.7 GiB VRAM, 51.8 GiB RAM, 15.3 cores.
Our fresh-issues benchmark: 14/50 (28 %) vs Strata IQ3_S 48/50, Q2_0 46/50, 27B 36/50 — 14 parse failures, 72 output truncations, 14 repetition hits, 2 agent format errors over 3,084 tool calls.
SWE-bench-Lite compare30: 21/30 (vs 28–30 for the GSQ-RCO quants), 5 parse failures, 18 truncations.
Needle: passes to 104,552 tokens.
Reading: the uncensored fine-tune (or its compat-BF16 repack on Strata) loses agentic reliability — runaway/looping replies and malformed tool calls — not speed. Not separable here: we did not run this GGUF on another engine.
Runs & Data › Decisions
T47 — Registry proofs: Strata Flash-Next IQ3_S and Q2_0 on RTX 3090 (operator-approved plan 2026-10-11) tickets/T47-registry-strata-3090.md
T47 — Registry proofs: Strata Flash-Next IQ3_S and Q2_0 on RTX 3090 (operator-approved plan 2026-10-11)
Host-kind recipes for sybil-solutions/local-ai-registry (precedent: #205), served from our container built from Strata's Dockerfile (pinned d5ea713). tools/registry/strata_proof.sh runs the registry's lab.py try --on endpoint in the registry clone (~/dev/local-ai-exp/work/registry-strata-3090, branch rtx-3090-strata-flash-next), sets needs.host_ram_gb to the measured peak and re-proves so launch_sha256 matches. PR body: prs/registry-strata-3090.md.
Result
IQ3_S: 6/6 gates, 75.0 tok/s, prefill 2,174, needs.host_ram_gb 57.0 (peak 56.88), recipe qwen3.8-flash-next.strata.128k.json, launch_sha256 215f1674…
Q2_0: 6/6 gates, 99.7 tok/s, prefill 2,664, needs.host_ram_gb 41.0 (peak 40.44), recipe qwen3.8-flash-next.strata-q2.128k.json, launch_sha256 27cc9992…
Runs & Data › Decisions
T48 — Qwen3.8-27B with the registry's 208k RTX 3090 recipe on our fresh-issues benchmark tickets/T48-27b-registry-208k-fresh.md
T48 — Qwen3.8-27B with the registry's 208k RTX 3090 recipe on our fresh-issues benchmark
Fairer 27B entry: the exact registry recipe (image c8922bd7…, 212,992 ctx, MTP, vision) instead of our 131k launch. Steps: lab, 10-min agentic stability, fresh-2026-09 test slice (same harness/sampling as T43). Exploratory vs the predeclared primary (which stays as run in T43).
Result (2026-10-12)
Registry 208k recipe: lab C1 126.3–138.7, prefill 831–836, agentic floor 90.1 (stability pass, 0 repetition hits). Fresh-issues benchmark: 39/50 (27 truncations, 43 repetition hits, 1 parse failure; 112 tok/s). Paired: Strata IQ3_S 48 vs 27B-208k 39 → discordant 10 vs 1, exact McNemar p = 0.0117 (exploratory). 27B-208k vs our 131k launch (36/50): 5 vs 2, p = 0.45 (no difference shown).
Runs & Data
Decisions and tickets Every non-trivial choice is written down before or right after it is made, most with an independent model review attached.
0001 — Flash-Next llama.cpp comparison quant (T11) 0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22) 0003 — Harness interpretations of SPEC §5 (repetition, parse failures) 0004 — Quarantine CPU core 2 (logical CPUs 2 and 18) 0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe) 0006 — PCIe lane layout: what can be reconfigured, and what it buys 0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs 0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate 0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision) 0010 — CPU boost off for the rest of the campaign 0011 — Flash-Next tuning at x8 (T14): grow the expert cache 0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26) 0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6) 0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05) 0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08) 0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09") 0017 — Site structure: story-led home, findings, recipes, model cards T41 — Strata (Flash-Next GGUF) screen on GPU2 (decision 0015 item 3) T42 — moetier on omarchy-gpu (decision 0015 item 2) T43 — Runs on the private fresh-2026-09 slice (decision 0016) T44 — Resource profiles: RAM, CPU, VRAM per phase (operator request 2026-10-10) T45 — Strata Coder (Qwen3.8-Flash-Next Coder, GSQ-RCO IQ1_M): full suite (operator request 2026-10-11) T46 — OrcaRouter Qwen3.8-Flash-Next-Uncensored IQ3_XXS on Strata (operator request 2026-10-11) T47 — Registry proofs: Strata Flash-Next IQ3_S and Q2_0 on RTX 3090 (operator-approved plan 2026-10-11) T48 — Qwen3.8-27B with the registry's 208k RTX 3090 recipe on our fresh-issues benchmark Qwen3.8-Flash-Next configs Model card →
One row per config_id, aggregating all of its records. Decode/prefill are the best value from a passing record (from a failed one when none passed, marked † and never counted as meeting a target). ✓ = meets the target (decode ≥ 50 tok/s, prefill > 1000 tok/s). Chips link to each run; ! = fail, ✕ = crash, – = skipped.
Config Lab decode tok/s Lab prefill tok/s TTFT prefill tok/s Kit C1 tok/s Occupied max Ctx configured Quality Stability Tool calls Engine Quant Cards Layout · PCIe Eligibility Runs strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb 103 ✓ 2,664 ✓ – – – 131k – – – strata GGUF Q2_0 1 current · x8 none load load lab lab lab lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 ! 1 fail 100 ✓ 2,664 ✓ 2,620 – 125k 131k – screened 1 run · min win 123 116 ✓ strata GGUF Q2_0 1 current · x8 none load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 93.2 ✓ 2,657 ✓ 2,639 – 106k 131k – – – strata GGUF IQ1_M 1 current · x8 none load lab TTFT strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 78.1 ✓ 2,178 ✓ 2,036 – 123k 131k – screened 1 run · min win 87.3 77 ✕ strata GGUF IQ3_S 1 current · x8 none load load load load lab lab TTFT stability quality eval quality eval resources strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 ! 1 fail 76.4 ✓ 2,122 ✓ 2,007 – 128k 131k – failed 1 run · min win 86.0 50 ✕ strata GGUF IQ3_XXS 1 current · x8 none load lab lab TTFT stability ! quality eval quality eval needle resources strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb 75.0 ✓ 2,174 ✓ – – – 131k – – – strata GGUF IQ3_S 1 current · x8 none load lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 ✕ 2 crash ! 1 fail 57.4 ✓ 2,537 ✓ 2,021 47.8 167k 205k in band top-1 0.991 · KL 0.00095858 failed 3 runs · min win 39.7 77 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 ! 1 fail 57.1 ✓ 608 ✕ 987 – 106k 131k – failed 1 run · min win 38.6 70 ✕ llama.cpp GGUF IQ4_XS 3 current · 3× x8 none load lab lab TTFT stability ! sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ! 5 fail 56.4 ✓ 2,240 ✓ 2,294 48.2 191k 205k in band top-1 0.99167 · KL 0.0009967 failed 4 runs · min win 38.8 256 ✓ sglang EXL3 3.05 bpw 1 current · x8 none load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 ! 1 fail 56.1 ✓ 2,222 ✓ – 48.5 167k 205k – failed 1 run · min win 39.0 53 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load lab lab kit sweep stability ! sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 ! 1 fail 55.6 ✓ 2,596 ✓ 2,017 48.2 167k 205k in band top-1 0.98867 · KL 0.00096647 failed 1 run · min win 37.9 67 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load lab lab kit sweep TTFT stability ! quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 ! 1 fail 55.6 ✓ 2,580 ✓ 2,014 – 167k 205k – failed 1 run · min win 40.1 57 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load lab lab TTFT stability ! fn-trellis-3.05-0xsero-main-x8-gpu2 ! 1 fail 55.3 ✓ 2,228 ✓ 2,263 46.9 167k 205k in band top-1 0.99234 · KL 0.0009667 failed 1 run · min win 38.1 63 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load lab lab kit sweep TTFT stability ! quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 ! 1 fail 53.7 ✓ 2,596 ✓ 2,022 – 167k 205k – failed 1 run · min win 40.9 72 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load lab lab TTFT stability ! sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 ! 1 fail 49.7 ✕ 2,564 ✓ 1,969 43.7 167k 205k in band top-1 0.98834 · KL 0.00099656 failed 1 run · min win 37.4 71 ✕ sglang EXL3 3.05 bpw 1 current · x8 none load lab lab kit sweep TTFT stability ! quality panel llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 44.3 ✕ 717 ✕ 1,067 – 106k 131k – – – llama.cpp GGUF IQ4_XS 3 current · 3× x8 none load lab TTFT llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 35.0 ✕ 713 ✕ 980 – 106k 131k – – – llama.cpp GGUF IQ4_XS 3 current · 3× x8 none load lab TTFT fn-llamacpp-iq4xs-128k-1x-fit-gpu2 ! 1 fail 26.5 ✕ 157 ✕ 269 – 106k 131k – failed 1 run · min win 23.8 49 ✕ llama.cpp GGUF IQ4_XS 1 current · x8 none load lab lab TTFT stability ! fn-llamacpp-iq4xs-128k-1x-gpu2 ✕ 1 crash – – – – – 131k – – – llama.cpp GGUF IQ4_XS 1 current · x8 none load ✕ llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 ! 1 fail – – – – – 131k – failed 1 run 0 ✕ llama.cpp GGUF IQ4_XS 3 current · 3× x8 none load stability ! sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2 ✕ 1 crash – – – – – 205k – – – sglang EXL3 2.05 bpw 1 current · x8 none load ✕ sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2 ✕ 1 crash – – – – – 205k – – – sglang EXL3 4.05 bpw 1 current · x8 none load ✕ sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 ✕ 2 crash – – – – – 205k – – – sglang EXL3 3.05 bpw 1 current · x8 none load lab ✕ TTFT ✕
GLM-5.3-Flash configs Model card →
One row per config_id, aggregating all of its records. Decode/prefill are the best value from a passing record (from a failed one when none passed, marked † and never counted as meeting a target). ✓ = meets the target (decode ≥ 15 tok/s). Chips link to each run; ! = fail, ✕ = crash, – = skipped.
Qwen3.8-27B configs Model card →
One row per config_id, aggregating all of its records. Decode/prefill are the best value from a passing record (from a failed one when none passed, marked † and never counted as meeting a target). ✓ = meets the target (decode ≥ 15 tok/s). Chips link to each run; ! = fail, ✕ = crash, – = skipped.
Superseded. This page is kept for old links; each chart now sits in the finding it supports.
Charts Each chart appears once records contain its measurements. Hover or tap a mark for its run; every chart has a table view.
Agentic floor by config The slowest 60-second window of decode during a 10-minute agentic coding run: what a coding agent actually gets. Best completed 10-minute run per model, quant and engine (runs below the 50 tok/s target are recorded as failed but count here).
Qwen3.8-Flash-Next GLM-5.3-Flash Qwen3.8-27B
0 50 100 150 Flash-Next GGUF Q2_0 · strata strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2: slowest 60 s window 123 tok/s, 116 tool calls 123 27B EXL3 3.00 bpw · sglang-exl3 sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2: slowest 60 s window 111 tok/s, 137 tool calls 111 Flash-Next GGUF IQ3_S · strata strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2: slowest 60 s window 87.3 tok/s, 77 tool calls 87.3 Flash-Next GGUF IQ3_XXS · strata strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2: slowest 60 s window 86.0 tok/s, 50 tool calls 86.0 Flash-Next EXL3 3.05 bpw · trellis/SGLang sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2: slowest 60 s window 40.9 tok/s, 72 tool calls 40.9 Flash-Next GGUF IQ4_XS · llama.cpp · 3 cards llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: slowest 60 s window 38.6 tok/s, 70 tool calls 38.6 Flash-Next GGUF IQ4_XS · llama.cpp fn-llamacpp-iq4xs-128k-1x-fit-gpu2: slowest 60 s window 23.8 tok/s, 49 tool calls 23.8 GLM GGUF IQ2_M · ik_llama.cpp · 3 cards ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: slowest 60 s window 13.3 tok/s, 0 tool calls 13.3 GLM GGUF IQ2_M · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012: slowest 60 s window 11.4 tok/s, 26 tool calls 11.4 GLM GGUF IQ3_XXS · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012: slowest 60 s window 9.86 tok/s, 22 tool calls 9.86 GLM GGUF IQ2_M · ik_llama.cpp ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2: slowest 60 s window 9.46 tok/s, 0 tool calls 9.46 GLM GGUF IQ2_M · llama.cpp glm-llamacpp-iq2m-128k-1x-gpu2: slowest 60 s window 8.91 tok/s, 24 tool calls 8.91 GLM GGUF Q2_K · llama.cpp llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2: slowest 60 s window 8.80 tok/s, 19 tool calls 8.80 GLM GGUF IQ4_XS · llama.cpp · 3 cards llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012: slowest 60 s window 7.81 tok/s, 25 tool calls 7.81 target 50 tok/s, slowest 60-second window of a 10-minute agentic coding run Table view Config Floor tok/s Run Flash-Next GGUF Q2_0 · strata 123 strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 27B EXL3 3.00 bpw · sglang-exl3 111 sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Flash-Next GGUF IQ3_S · strata 87.3 strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Flash-Next GGUF IQ3_XXS · strata 86.0 strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Flash-Next EXL3 3.05 bpw · trellis/SGLang 40.9 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Flash-Next GGUF IQ4_XS · llama.cpp · 3 cards 38.6 llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Flash-Next GGUF IQ4_XS · llama.cpp 23.8 fn-llamacpp-iq4xs-128k-1x-fit-gpu2 GLM GGUF IQ2_M · ik_llama.cpp · 3 cards 13.3 ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 GLM GGUF IQ2_M · llama.cpp · 3 cards 11.4 llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 GLM GGUF IQ3_XXS · llama.cpp · 3 cards 9.86 llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 GLM GGUF IQ2_M · ik_llama.cpp 9.46 ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 GLM GGUF IQ2_M · llama.cpp 8.91 glm-llamacpp-iq2m-128k-1x-gpu2 GLM GGUF Q2_K · llama.cpp 8.80 llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 GLM GGUF IQ4_XS · llama.cpp · 3 cards 7.81 llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 GLM GGUF IQ3_XXS · llama.cpp 7.65 llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 GLM EXL3 3.05 bpw · exllamav3 3.36 exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
What limits each setup Host-side utilization sampled every 0.1 s during the run (moetier probe), as a share of this box's measured ceilings. The bar nearest 100 % is the bottleneck.
GPU busy PCIe host→GPU, % of measured 13.5 GB/s NVMe read, % of 3.36 GB/s CPU busy
0% 20% 40% 60% 80% 100% GLM-5.3 · NVMe recipe · 55 GiB RAM: GPU busy 87 % of samples with a kernel running 87% GLM-5.3 · NVMe recipe · 55 GiB RAM: PCIe in 3.34 GB/s of 13.5 (p95 6.17) 25% GLM-5.3 · NVMe recipe · 55 GiB RAM: NVMe read 3.08 GB/s of 3.36 92% GLM-5.3 · NVMe recipe · 55 GiB RAM: CPU 62% busy (expert-compute cores) 62% GLM-5.3 55 GiB GLM-5.3 · NVMe recipe · 90 GiB RAM: GPU busy 96 % of samples with a kernel running 96% GLM-5.3 · NVMe recipe · 90 GiB RAM: PCIe in 6.42 GB/s of 13.5 (p95 14.27) 48% GLM-5.3 · NVMe recipe · 90 GiB RAM: NVMe read 1.31 GB/s of 3.36 39% GLM-5.3 · NVMe recipe · 90 GiB RAM: CPU 83% busy (expert-compute cores) 83% GLM-5.3 90 GiB Flash-Next · trellis · agentic run: GPU busy 89 % of samples with a kernel running 89% Flash-Next · trellis · agentic run: PCIe in 7.77 GB/s of 13.5 (p95 14.25) 58% Flash-Next · trellis · agentic run: NVMe read 0.0 GB/s of 3.36 0% Flash-Next · trellis · agentic run: CPU 4% busy (all cores) 4% Flash-Next agentic share of ceiling
Table view Arm Phase GPU PCIe NVMe CPU GLM-5.3 · NVMe recipe · 55 GiB RAM kit 87 % of samples with a kernel running 3.34 GB/s of 13.5 (p95 6.17) 3.08 GB/s of 3.36 62% busy (expert-compute cores) GLM-5.3 · NVMe recipe · 90 GiB RAM kit 96 % of samples with a kernel running 6.42 GB/s of 13.5 (p95 14.27) 1.31 GB/s of 3.36 83% busy (expert-compute cores) Flash-Next · trellis · agentic run stab 89 % of samples with a kernel running 7.77 GB/s of 13.5 (p95 14.25) 0.0 GB/s of 3.36 4% busy (all cores)
Decode vs occupied context Lab C1 decode and the stability minimum 60 s window against the occupied prompt tokens reported by the server.
Qwen3.8-Flash-Next GLM-5.3-Flash Qwen3.8-27B lab C1 (first 30 s) stability: min 60 s window0 50 100 150 200 0 50 100 150 200 occupied context (k tokens) decode tok/s Flash-Next target 50 GLM gate 15 128k fn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 49.3 tok/s at 166,667 occupied · pass fn-trellis-3.05-0xsero-main-x8-gpu2 · stability min 60 s window 38.1 tok/s at 30,034 occupied · fail fn-trellis-3.05-0xsero-main-x8-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · pass fn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.5 tok/s at 106,294 occupied · pass fn-llamacpp-iq4xs-128k-1x-fit-gpu2 · stability min 60 s window 23.8 tok/s at 16,819 occupied · fail fn-llamacpp-iq4xs-128k-1x-fit-gpu2 · lab C1 decode 26.1 tok/s at 106,294 occupied · pass glm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail glm-llamacpp-iq2m-128k-1x-gpu2 · stability min 60 s window 8.91 tok/s at 9,047 occupied · fail glm-llamacpp-iq2m-128k-1x-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.20 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · stability min 60 s window 8.80 tok/s at 6,658 occupied · fail llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 · lab C1 decode 9.30 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 10.0 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · stability min 60 s window 9.46 tok/s at 1,874 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 · lab C1 decode 8.20 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.30 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · stability min 60 s window 7.65 tok/s at 4,875 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 · lab C1 decode 8.40 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · stability min 60 s window 8.85 tok/s at 9,732 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 · lab C1 decode 8.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · lab C1 decode 12.3 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 · stability min 60 s window 11.4 tok/s at 8,118 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.5 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · stability min 60 s window 9.86 tok/s at 5,309 occupied · fail llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 · lab C1 decode 10.4 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 · lab C1 decode 12.2 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 11.0 tok/s at 6,174 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 11.8 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 · lab C1 decode 9.50 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · stability min 60 s window 8.85 tok/s at 6,632 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 · lab C1 decode 9.60 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.00 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · stability min 60 s window 7.81 tok/s at 6,852 occupied · fail llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 · lab C1 decode 8.30 tok/s at 93,437 occupied · fail llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 56.9 tok/s at 106,294 occupied · pass llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · stability min 60 s window 38.6 tok/s at 39,226 occupied · fail llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 · lab C1 decode 57.1 tok/s at 106,294 occupied · pass llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 · lab C1 decode 9.10 tok/s at 93,437 occupied · fail llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 · lab C1 decode 11.2 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 14.0 tok/s at 93,437 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · stability min 60 s window 13.3 tok/s at 1,874 occupied · fail ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 · lab C1 decode 13.4 tok/s at 93,437 occupied · fail llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 · lab C1 decode 44.3 tok/s at 106,294 occupied · pass llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 · lab C1 decode 35.0 tok/s at 106,294 occupied · pass ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 · lab C1 decode 10.2 tok/s at 93,437 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · stability min 60 s window 37.9 tok/s at 36,327 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · lab C1 decode 54.5 tok/s at 166,667 occupied · pass exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.40 tok/s at 93,437 occupied · fail exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · stability min 60 s window 3.36 tok/s at 4,589 occupied · fail exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 · lab C1 decode 7.50 tok/s at 93,437 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 55.8 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · stability min 60 s window 39.7 tok/s at 25,309 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 57.4 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 55.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · stability min 60 s window 40.1 tok/s at 29,027 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 53.7 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · stability min 60 s window 40.9 tok/s at 39,281 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 · lab C1 decode 51.3 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 48.1 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · stability min 60 s window 37.4 tok/s at 34,417 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · lab C1 decode 49.7 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.9 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.5 tok/s at 25,179 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 49.5 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · lab C1 decode 49.9 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 53.5 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 38.8 tok/s at 28,209 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 56.4 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 55.0 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 40.4 tok/s at 30,539 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 45.0 tok/s at 166,667 occupied · pass sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 · stability min 60 s window 111 tok/s at 52,045 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 48.8 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · stability min 60 s window 39.8 tok/s at 28,542 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · lab C1 decode 54.6 tok/s at 166,667 occupied · pass sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 · lab C1 decode 144 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 123 tok/s at 38,712 occupied · pass strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 · lab C1 decode 100 tok/s at 106,294 occupied · pass glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 · lab C1 decode 6.10 tok/s at 93,437 occupied · fail glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 · lab C1 decode 11.3 tok/s at 93,437 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 55.3 tok/s at 166,667 occupied · pass sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · stability min 60 s window 39.0 tok/s at 28,256 occupied · fail sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 · lab C1 decode 56.1 tok/s at 166,667 occupied · pass strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 78.1 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 87.3 tok/s at 39,624 occupied · pass strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 · lab C1 decode 75.5 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 · lab C1 decode 93.2 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 76.4 tok/s at 106,294 occupied · pass strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · stability min 60 s window 86.0 tok/s at 56,311 occupied · fail strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 · lab C1 decode 73.6 tok/s at 106,294 occupied · pass sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 126 tok/s at 173,372 occupied · pass sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · stability min 60 s window 90.1 tok/s at 68,013 occupied · pass sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 · lab C1 decode 139 tok/s at 173,372 occupied · pass Table view Model Config Measure Occupied tok/s Outcome Qwen3.8-Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 lab C1 decode 166,667 49.3 ✓ pass Qwen3.8-Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 stability min 60 s window 30,034 38.1 ! fail Qwen3.8-Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 lab C1 decode 166,667 55.3 ✓ pass Qwen3.8-Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 lab C1 decode 106,294 26.5 ✓ pass Qwen3.8-Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 stability min 60 s window 16,819 23.8 ! fail Qwen3.8-Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 lab C1 decode 106,294 26.1 ✓ pass GLM-5.3-Flash glm-llamacpp-iq2m-128k-1x-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash glm-llamacpp-iq2m-128k-1x-gpu2 stability min 60 s window 9,047 8.91 ! fail GLM-5.3-Flash glm-llamacpp-iq2m-128k-1x-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 lab C1 decode 93,437 9.20 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 stability min 60 s window 6,658 8.80 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 lab C1 decode 93,437 9.30 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 lab C1 decode 93,437 10.0 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 stability min 60 s window 1,874 9.46 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 lab C1 decode 93,437 8.20 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 lab C1 decode 93,437 8.30 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 stability min 60 s window 4,875 7.65 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 lab C1 decode 93,437 8.40 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 stability min 60 s window 9,732 8.85 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 lab C1 decode 93,437 8.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 lab C1 decode 93,437 12.3 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 stability min 60 s window 8,118 11.4 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 lab C1 decode 93,437 10.5 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 stability min 60 s window 5,309 9.86 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 lab C1 decode 93,437 10.4 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 lab C1 decode 93,437 12.2 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 11.8 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 stability min 60 s window 6,174 11.0 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 11.8 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 lab C1 decode 93,437 9.50 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 stability min 60 s window 6,632 8.85 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 lab C1 decode 93,437 9.60 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 8.00 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 stability min 60 s window 6,852 7.81 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 lab C1 decode 93,437 8.30 ! fail Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 lab C1 decode 106,294 56.9 ✓ pass Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 stability min 60 s window 39,226 38.6 ! fail Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 lab C1 decode 106,294 57.1 ✓ pass GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 lab C1 decode 93,437 9.10 ! fail GLM-5.3-Flash llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 lab C1 decode 93,437 11.2 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 lab C1 decode 93,437 14.0 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 stability min 60 s window 1,874 13.3 ! fail GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 lab C1 decode 93,437 13.4 ! fail Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 lab C1 decode 106,294 44.3 ✓ pass Qwen3.8-Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 lab C1 decode 106,294 35.0 ✓ pass GLM-5.3-Flash ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 lab C1 decode 93,437 10.2 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 lab C1 decode 166,667 55.6 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 stability min 60 s window 36,327 37.9 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 lab C1 decode 166,667 54.5 ✓ pass GLM-5.3-Flash exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 lab C1 decode 93,437 7.40 ! fail GLM-5.3-Flash exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 stability min 60 s window 4,589 3.36 ! fail GLM-5.3-Flash exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 lab C1 decode 93,437 7.50 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab C1 decode 166,667 55.8 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 stability min 60 s window 25,309 39.7 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab C1 decode 166,667 57.4 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 lab C1 decode 166,667 55.6 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 stability min 60 s window 29,027 40.1 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 lab C1 decode 166,667 54.6 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 lab C1 decode 166,667 53.7 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 stability min 60 s window 39,281 40.9 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 lab C1 decode 166,667 51.3 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 lab C1 decode 166,667 48.1 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 stability min 60 s window 34,417 37.4 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 lab C1 decode 166,667 49.7 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 54.9 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 25,179 39.5 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 49.5 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab C1 decode 166,667 49.9 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 53.5 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 28,209 38.8 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 56.4 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 55.0 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 30,539 40.4 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 45.0 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 stability min 60 s window 52,045 111 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 48.8 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 stability min 60 s window 28,542 39.8 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab C1 decode 166,667 54.6 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 lab C1 decode 106,294 144 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 100 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 stability min 60 s window 38,712 123 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 100 ✓ pass GLM-5.3-Flash glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 lab C1 decode 93,437 6.10 ! fail GLM-5.3-Flash glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 lab C1 decode 93,437 11.3 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 lab C1 decode 166,667 55.3 ✓ pass Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 stability min 60 s window 28,256 39.0 ! fail Qwen3.8-Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 lab C1 decode 166,667 56.1 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 78.1 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 stability min 60 s window 39,624 87.3 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 75.5 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 93.2 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 76.4 ✓ pass Qwen3.8-Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 stability min 60 s window 56,311 86.0 ! fail Qwen3.8-Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 lab C1 decode 106,294 73.6 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 lab C1 decode 173,372 126 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 stability min 60 s window 68,013 90.1 ✓ pass Qwen3.8-27B sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 lab C1 decode 173,372 139 ✓ pass
Speed vs quality Best lab decode per config against its quality measurement.
Qwen3.8-Flash-Next: decode vs mean KL (lower KL is better) 0 20 40 60 80 0 0.0005 0.001 0.0015 mean KL vs exllamav3 reference panel lab decode tok/s target 50 band KL ≤ 0.0013 fn-trellis-3.05-0xsero-main-x8-gpu2 · KL 0.0009667 · top-1 0.99234 · decode 55.3 fn-trellis-3.05-0xsero-main-x8-gpu2 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 · KL 0.00096647 · top-1 0.98867 · decode 55.6 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 · KL 0.00095858 · top-1 0.991 · decode 57.4 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 · KL 0.00099656 · top-1 0.98834 · decode 49.7 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 · KL 0.0009967 · top-1 0.99167 · decode 56.4 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
Table view Config KL top-1 Lab decode fn-trellis-3.05-0xsero-main-x8-gpu2 0.0009667 0.99234 55.3 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 0.00096647 0.98867 55.6 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 0.00095858 0.991 57.4 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 0.00099656 0.98834 49.7 sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 0.0009967 0.99167 56.4
1 vs 3 cards (current layout) Best config per card count, same model.
Best lab decode 0 50 100 150 Flash-Next GGUF Q2_0 · 1 card strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb: 103 tok/s decode 103 Flash-Next GGUF IQ4_XS · 3 cards llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: 57.1 tok/s decode 57.1 GLM ? · 1 card glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2: 11.3 tok/s decode 11.3 GLM GGUF IQ2_M · 3 cards ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: 14.0 tok/s decode 14.0 tok/s Lab prefill of the same configs 0 1k 2k 3k Flash-Next GGUF Q2_0 · 1 card strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb: 2,664 tok/s prefill 2,664 Flash-Next GGUF IQ4_XS · 3 cards llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012: 608 tok/s prefill 608 GLM ? · 1 card glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2: 630 tok/s prefill 630 GLM GGUF IQ2_M · 3 cards ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012: 189 tok/s prefill 189 tok/s Table view x8 vs x16 No measurements yet: needs single-card lab results in both layouts for a model.
All runs Every record, newest first, failed, crashed and skipped runs included.
Started Outcome Model Kind Config Key metrics Stage Ticket 2026-10-12 11:48 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb lab_decode_c1_tps 99.7 tok/s · lab_prefill_tps 2,664 tok/s T47 2026-10-12 11:48 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb load_seconds 25.0 s T47 2026-10-12 05:49 AEST ✓ pass 27B quality eval sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 solve_rate 0.78 fraction T48 2026-10-12 05:44 AEST ✓ pass 27B lab sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 139 tok/s · lab_prefill_tps 836 tok/s T48 2026-10-12 05:33 AEST ✓ pass 27B stability sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 decode_window_min_tps 90.1 tok/s · tool_calls 134 calls T48 2026-10-12 05:28 AEST ✓ pass 27B lab sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 126 tok/s · lab_prefill_tps 831 tok/s T48 2026-10-12 05:26 AEST ✓ pass 27B load sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 load_seconds 113 s T48 2026-10-12 05:23 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb lab_decode_c1_tps 88.2 tok/s · lab_prefill_tps 2,638 tok/s T47 2026-10-12 05:20 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb lab_decode_c1_tps 101 tok/s · lab_prefill_tps 2,631 tok/s T47 2026-10-12 05:18 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb lab_decode_c1_tps 103 tok/s · lab_prefill_tps 2,644 tok/s T47 2026-10-12 05:17 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb load_seconds 25.0 s T47 2026-10-12 05:15 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb lab_decode_c1_tps 75.0 tok/s · lab_prefill_tps 2,174 tok/s T47 2026-10-12 05:14 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb load_seconds 34.1 s T47 2026-10-12 00:44 AEST ✓ pass Flash-Next quality eval strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 solve_rate 0.7 fraction T46 2026-10-12 00:41 AEST ✓ pass Flash-Next needle strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 T46 2026-10-11 12:13 AEST ✓ pass Flash-Next quality eval strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 solve_rate 0.28 fraction T46 2026-10-11 12:11 AEST ✓ pass Flash-Next resources strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 T46 2026-10-11 12:06 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 73.6 tok/s · lab_prefill_tps 2,105 tok/s T46 2026-10-11 11:56 AEST ! fail Flash-Next stability strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 decode_window_min_tps 86.0 tok/s · tool_calls 50 calls T46 2026-10-11 11:55 AEST ✓ pass Flash-Next TTFT strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 ttft_prefill_tps 2,007 tok/s T46 2026-10-11 11:48 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 76.4 tok/s · lab_prefill_tps 2,122 tok/s T46 2026-10-11 11:47 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 load_seconds 34.1 s T46 2026-10-11 10:50 AEST ✓ pass Flash-Next TTFT strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 ttft_prefill_tps 2,639 tok/s T45 2026-10-11 10:48 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 93.2 tok/s · lab_prefill_tps 2,657 tok/s T45 2026-10-11 10:47 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 load_seconds 28.1 s T45 2026-10-10 21:00 AEST ✓ pass Flash-Next resources sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 T44 2026-10-10 20:56 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 264 s T44 2026-10-10 20:53 AEST ✓ pass 27B resources sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 T44 2026-10-10 20:51 AEST ✓ pass 27B load sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 load_seconds 131 s T44 2026-10-10 20:48 AEST ✓ pass Flash-Next resources strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 T44 2026-10-10 20:48 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 load_seconds 31.1 s T44 2026-10-10 20:46 AEST ✓ pass Flash-Next resources strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 T44 2026-10-10 20:46 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 load_seconds 25.1 s T44 2026-10-10 10:23 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 267 s T43 2026-10-10 03:48 AEST ✓ pass Flash-Next quality eval strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 solve_rate 0.92 fraction T43 2026-10-10 03:48 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 load_seconds 24.9 s T43 2026-10-09 21:38 AEST ✓ pass 27B quality eval sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 solve_rate 0.72 fraction T43 2026-10-09 21:36 AEST ✓ pass 27B load sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 load_seconds 131 s T43 2026-10-09 11:02 AEST ✓ pass Flash-Next quality eval strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 solve_rate 0.96 fraction T43 2026-10-09 11:01 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 load_seconds 31.0 s T43 2026-10-09 02:28 AEST ✓ pass Flash-Next quality eval strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 solve_rate 0.9667 fraction T41 2026-10-09 02:28 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 load_seconds 30.9 s T41 2026-10-08 23:43 AEST ✓ pass Flash-Next quality eval strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 solve_rate 0.9333 fraction T41 2026-10-08 23:43 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 load_seconds 24.9 s T41 2026-10-08 23:39 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 75.5 tok/s · lab_prefill_tps 2,178 tok/s T41 2026-10-08 23:28 AEST ✓ pass Flash-Next stability strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 decode_window_min_tps 87.3 tok/s · tool_calls 77 calls T41 2026-10-08 23:27 AEST ✓ pass Flash-Next TTFT strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 ttft_prefill_tps 2,036 tok/s T41 2026-10-08 23:23 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 78.1 tok/s · lab_prefill_tps 2,174 tok/s T41 2026-10-08 23:22 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 load_seconds 43.0 s T41 2026-10-08 23:05 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 lab_decode_c1_tps 56.1 tok/s · lab_prefill_tps 2,222 tok/s T42 2026-10-08 22:55 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 decode_window_min_tps 39.0 tok/s · tool_calls 53 calls T42 2026-10-08 22:40 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 decode_c1_tps 48.5 tok/s · prefill_32k_tps 2,435 tok/s T42 2026-10-08 22:36 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 lab_decode_c1_tps 55.3 tok/s · lab_prefill_tps 2,107 tok/s T42 2026-10-08 22:31 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 load_seconds 260 s T42 2026-10-08 22:26 AEST ✓ pass GLM TTFT glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 ttft_prefill_tps 336 tok/s T42 2026-10-08 22:23 AEST ✓ pass GLM quality panel glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 top1_agreement 1.00 fraction · mean_kl 0 nats T42 2026-10-08 20:52 AEST ✓ pass GLM kit sweep glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 decode_c1_tps 10.9 tok/s · prefill_32k_tps 621 tok/s T42 2026-10-08 20:38 AEST ! fail GLM lab glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 lab_decode_c1_tps 11.3 tok/s · lab_prefill_tps 630 tok/s gates T42 2026-10-08 20:36 AEST ✓ pass GLM load glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 load_seconds 113 s T42 2026-10-08 20:25 AEST ✓ pass GLM TTFT glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 ttft_prefill_tps 193 tok/s T42 2026-10-08 19:34 AEST ✓ pass GLM quality panel glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 top1_agreement 1.00 fraction · mean_kl 0 nats T42 2026-10-08 17:34 AEST ! fail GLM kit sweep glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 decode_c1_tps unavailable · prefill_32k_tps unavailable timeout T42 2026-10-08 16:34 AEST ! fail GLM lab glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 lab_decode_c1_tps 6.10 tok/s · lab_prefill_tps 314 tok/s gates T42 2026-10-08 16:32 AEST ✓ pass GLM load glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 load_seconds 101 s T42 2026-10-08 16:29 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 100 tok/s · lab_prefill_tps 2,657 tok/s T41 2026-10-08 16:18 AEST ✓ pass Flash-Next stability strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 decode_window_min_tps 123 tok/s · tool_calls 116 calls T41 2026-10-08 16:17 AEST ✓ pass Flash-Next TTFT strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 ttft_prefill_tps 2,620 tok/s T41 2026-10-08 16:17 AEST ! fail Flash-Next kit sweep strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 decode_c1_tps unavailable · prefill_32k_tps unavailable measure T41 2026-10-08 16:14 AEST ✓ pass Flash-Next lab strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 lab_decode_c1_tps 100 tok/s · lab_prefill_tps 2,664 tok/s T41 2026-10-08 16:14 AEST ✓ pass Flash-Next load strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 load_seconds 24.9 s T41 2026-10-06 03:41 AEST ! fail 27B needle sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 measure T40 2026-10-06 03:37 AEST ✓ pass 27B lab sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 lab_decode_c1_tps 144 tok/s · lab_prefill_tps 740 tok/s T40 2026-10-06 03:35 AEST ✓ pass 27B load sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 load_seconds 128 s T40 2026-10-06 03:30 AEST ✓ pass Flash-Next needle sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 T40 2026-10-06 03:25 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 260 s T40 2026-10-06 03:18 AEST ! fail 27B needle sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 measure T40 2026-10-06 03:16 AEST ✓ pass 27B load sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 load_seconds 128 s T40 2026-10-06 03:15 AEST ! fail 27B lab sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 lab_decode_c1_tps unavailable · lab_prefill_tps unavailable gates T40 2026-10-06 03:13 AEST ✓ pass 27B load sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 load_seconds 128 s T40 2026-10-05 21:50 AEST ✓ pass Flash-Next quality eval sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 solve_rate 1.00 fraction T40 2026-10-05 21:45 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 54.6 tok/s · lab_prefill_tps 2,234 tok/s T40 2026-10-05 21:35 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_window_min_tps 39.8 tok/s · tool_calls 69 calls T40 2026-10-05 21:32 AEST ! fail Flash-Next needle sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 measure T40 2026-10-05 21:31 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ttft_prefill_tps 2,284 tok/s T40 2026-10-05 21:16 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_c1_tps 47.5 tok/s · prefill_32k_tps 2,444 tok/s T40 2026-10-05 21:10 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 48.8 tok/s · lab_prefill_tps 2,131 tok/s T40 2026-10-05 21:06 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 263 s T40 2026-10-05 18:36 AEST ✓ pass 27B quality eval sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 solve_rate 0.6 fraction T40 2026-10-05 18:35 AEST ! fail 27B lab sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 lab_decode_c1_tps unavailable · lab_prefill_tps unavailable gates T40 2026-10-05 18:25 AEST ! fail 27B stability sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 decode_window_min_tps 111 tok/s · tool_calls 137 calls T40 2026-10-05 18:25 AEST ! fail 27B needle sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 measure T40 2026-10-05 18:21 AEST ✓ pass 27B TTFT sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 ttft_prefill_tps 1,035 tok/s T40 2026-10-05 18:08 AEST ✓ pass 27B kit sweep sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 decode_c1_tps 121 tok/s · prefill_32k_tps 970 tok/s T40 2026-10-05 18:07 AEST ! fail 27B lab sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 lab_decode_c1_tps unavailable · lab_prefill_tps unavailable gates T40 2026-10-05 18:05 AEST ✓ pass 27B load sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 load_seconds 128 s T40 2026-10-05 12:24 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 45.0 tok/s · lab_prefill_tps 2,240 tok/s T14a 2026-10-05 12:13 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_window_min_tps 40.4 tok/s · tool_calls 61 calls T14a 2026-10-05 11:58 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_c1_tps 48.1 tok/s · prefill_32k_tps 2,434 tok/s T14a 2026-10-05 11:58 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 top1_agreement 0.99167 fraction · mean_kl 0.0009967 nats T14a 2026-10-05 11:56 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ttft_prefill_tps 2,289 tok/s T14a 2026-10-05 11:50 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 55.0 tok/s · lab_prefill_tps 2,131 tok/s T14a 2026-10-05 11:45 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 260 s T14a 2026-10-05 11:38 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 56.4 tok/s · lab_prefill_tps 2,228 tok/s T14a 2026-10-05 11:27 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_window_min_tps 38.8 tok/s · tool_calls 62 calls T14a 2026-10-05 11:27 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 top1_agreement 0.98934 fraction · mean_kl 0.00090651 nats T14a 2026-10-05 11:25 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ttft_prefill_tps 2,283 tok/s T14a 2026-10-05 11:10 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_c1_tps 47.0 tok/s · prefill_32k_tps 2,440 tok/s T14a 2026-10-05 11:03 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 53.5 tok/s · lab_prefill_tps 2,134 tok/s T14a 2026-10-05 10:59 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 257 s T14a 2026-10-05 10:49 AEST ✕ crash Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 decode_window_min_tps unavailable · tool_calls 6 calls measure T13 2026-10-05 10:48 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 top1_agreement 0.991 fraction · mean_kl 0.00095858 nats T13 2026-10-05 10:47 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 ttft_prefill_tps 2,021 tok/s T13 2026-10-05 10:32 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 decode_c1_tps 47.8 tok/s · prefill_32k_tps 2,669 tok/s T13 2026-10-05 10:25 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab_decode_c1_tps 49.9 tok/s · lab_prefill_tps 2,419 tok/s T13 2026-10-05 10:21 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 load_seconds 263 s T13 2026-10-05 10:17 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 49.5 tok/s · lab_prefill_tps 2,237 tok/s T14a 2026-10-05 10:07 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_window_min_tps 39.5 tok/s · tool_calls 64 calls T14a 2026-10-05 10:06 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 top1_agreement 0.98934 fraction · mean_kl 0.00091506 nats T14a 2026-10-05 10:05 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ttft_prefill_tps 2,294 tok/s T14a 2026-10-05 09:49 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 decode_c1_tps 48.2 tok/s · prefill_32k_tps 2,436 tok/s T14a 2026-10-05 09:41 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 lab_decode_c1_tps 54.9 tok/s · lab_prefill_tps 2,131 tok/s T14a 2026-10-05 09:37 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 load_seconds 260 s T14a 2026-10-05 09:33 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 lab_decode_c1_tps 49.7 tok/s · lab_prefill_tps 2,564 tok/s T14a 2026-10-05 09:22 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 decode_window_min_tps 37.4 tok/s · tool_calls 71 calls T14a 2026-10-05 09:21 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 top1_agreement 0.98834 fraction · mean_kl 0.00099656 nats T14a 2026-10-05 09:20 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 ttft_prefill_tps 1,969 tok/s T14a 2026-10-05 09:04 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 decode_c1_tps 43.7 tok/s · prefill_32k_tps 2,644 tok/s T14a 2026-10-05 09:00 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 lab_decode_c1_tps 48.1 tok/s · lab_prefill_tps 2,408 tok/s T14a 2026-10-05 08:55 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 load_seconds 260 s T14a 2026-10-05 00:18 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab_decode_c1_tps 55.8 tok/s · lab_prefill_tps 2,419 tok/s T13 2026-10-05 00:14 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 load_seconds 257 s T13 2026-10-05 00:07 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 lab_decode_c1_tps 51.3 tok/s · lab_prefill_tps 2,596 tok/s T14b 2026-10-04 23:57 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 decode_window_min_tps 40.9 tok/s · tool_calls 72 calls T14b 2026-10-04 23:55 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 ttft_prefill_tps 2,022 tok/s T14b 2026-10-04 23:52 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 lab_decode_c1_tps 53.7 tok/s · lab_prefill_tps 2,440 tok/s T14b 2026-10-04 23:48 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 load_seconds 215 s T14b 2026-10-04 23:43 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 lab_decode_c1_tps 54.6 tok/s · lab_prefill_tps 2,580 tok/s T14a 2026-10-04 23:33 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 decode_window_min_tps 40.1 tok/s · tool_calls 57 calls T14a 2026-10-04 23:31 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 ttft_prefill_tps 2,014 tok/s T14a 2026-10-04 23:26 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 lab_decode_c1_tps 55.6 tok/s · lab_prefill_tps 2,430 tok/s T14a 2026-10-04 23:22 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 load_seconds 254 s T14a 2026-10-04 23:13 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab_decode_c1_tps 57.4 tok/s · lab_prefill_tps 2,537 tok/s T14a 2026-10-04 23:02 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 decode_window_min_tps 39.7 tok/s · tool_calls 71 calls T14a 2026-10-04 22:58 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 load_seconds 264 s T14a 2026-10-04 22:54 AEST ✓ pass GLM TTFT exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 ttft_prefill_tps 483 tok/s T23 2026-10-04 22:49 AEST ✓ pass GLM load exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 load_seconds 276 s T23 2026-10-04 22:48 AEST ✕ crash Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 ttft_prefill_tps unavailable measure T14a 2026-10-04 22:47 AEST ✕ crash Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 lab_decode_c1_tps unavailable · lab_prefill_tps unavailable harness T14a 2026-10-04 22:42 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 load_seconds 257 s T14a 2026-10-04 22:32 AEST ✕ crash Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 decode_window_min_tps unavailable · tool_calls 0 calls measure T13 2026-10-04 22:32 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 top1_agreement 0.98967 fraction · mean_kl 0.00097327 nats T13 2026-10-04 22:30 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 ttft_prefill_tps 2,009 tok/s T13 2026-10-04 22:15 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 decode_c1_tps 47.4 tok/s · prefill_32k_tps 2,678 tok/s T13 2026-10-04 22:10 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 lab_decode_c1_tps 55.8 tok/s · lab_prefill_tps 2,419 tok/s T13 2026-10-04 22:05 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 load_seconds 260 s T13 2026-10-04 21:27 AEST ! fail GLM lab exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 lab_decode_c1_tps 7.50 tok/s · lab_prefill_tps 720 tok/s gates T23 2026-10-04 21:17 AEST ! fail GLM stability exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 decode_window_min_tps 3.36 tok/s · tool_calls 21 calls T23 2026-10-04 21:13 AEST ! fail GLM TTFT exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 ttft_prefill_tps unavailable invalid measurement T23 2026-10-04 21:02 AEST ! fail GLM lab exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 lab_decode_c1_tps 7.40 tok/s · lab_prefill_tps 739 tok/s gates T23 2026-10-04 20:57 AEST ✓ pass GLM load exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 load_seconds 300 s T23 2026-10-04 20:56 AEST ✕ crash Flash-Next load sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2 load_seconds unavailable load T15 2026-10-04 20:52 AEST ✕ crash Flash-Next load sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2 load_seconds unavailable load T15 2026-10-04 20:47 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 lab_decode_c1_tps 54.5 tok/s · lab_prefill_tps 2,596 tok/s T13 2026-10-04 20:36 AEST ! fail Flash-Next stability sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 decode_window_min_tps 37.9 tok/s · tool_calls 67 calls T13 2026-10-04 20:36 AEST ✓ pass Flash-Next quality panel sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 top1_agreement 0.98867 fraction · mean_kl 0.00096647 nats T13 2026-10-04 20:34 AEST ✓ pass Flash-Next TTFT sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 ttft_prefill_tps 2,017 tok/s T13 2026-10-04 20:19 AEST ✓ pass Flash-Next kit sweep sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 decode_c1_tps 48.2 tok/s · prefill_32k_tps 2,678 tok/s T13 2026-10-04 20:14 AEST ✓ pass Flash-Next lab sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 lab_decode_c1_tps 55.6 tok/s · lab_prefill_tps 2,433 tok/s T13 2026-10-04 20:10 AEST ✓ pass Flash-Next load sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 load_seconds 236 s T13 2026-10-04 19:38 AEST ✕ crash GLM stability ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 decode_window_min_tps unavailable · tool_calls 0 calls measure T22c 2026-10-04 19:26 AEST ✓ pass GLM TTFT ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 ttft_prefill_tps 248 tok/s T22c 2026-10-04 19:12 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 lab_decode_c1_tps 10.2 tok/s · lab_prefill_tps 166 tok/s gates T22c 2026-10-04 19:11 AEST ✓ pass GLM load ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 load_seconds 48.6 s T22c 2026-10-04 19:11 AEST ✓ pass none hw bw hw-bw-gpu1-current h2d_gbps 13.5 GB/s · d2h_gbps 13.2 GB/s T05a 2026-10-04 18:19 AEST ✓ pass Flash-Next TTFT llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 ttft_prefill_tps 980 tok/s T11 2026-10-04 18:11 AEST ✓ pass Flash-Next lab llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 lab_decode_c1_tps 35.0 tok/s · lab_prefill_tps 713 tok/s T11 2026-10-04 18:10 AEST ✓ pass Flash-Next load llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 load_seconds 39.9 s T11 2026-10-04 18:06 AEST ✓ pass Flash-Next TTFT llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 ttft_prefill_tps 1,067 tok/s T11 2026-10-04 18:01 AEST ✓ pass Flash-Next lab llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 lab_decode_c1_tps 44.3 tok/s · lab_prefill_tps 717 tok/s T11 2026-10-04 18:00 AEST ✓ pass Flash-Next load llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 load_seconds 39.9 s T11 2026-10-04 17:42 AEST ✕ crash GLM load ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012 load_seconds unavailable load T22c 2026-10-04 17:31 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 lab_decode_c1_tps 13.4 tok/s · lab_prefill_tps 189 tok/s gates T22c 2026-10-04 17:21 AEST ! fail GLM stability ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 decode_window_min_tps 13.3 tok/s · tool_calls 0 calls T22c 2026-10-04 17:10 AEST ✓ pass GLM TTFT ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 ttft_prefill_tps 283 tok/s T22c 2026-10-04 16:58 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 lab_decode_c1_tps 14.0 tok/s · lab_prefill_tps 185 tok/s gates T22c 2026-10-04 16:58 AEST ✓ pass GLM load ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 load_seconds 44.3 s T22c 2026-10-04 16:48 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 ttft_prefill_tps 298 tok/s T21 2026-10-04 16:30 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 lab_decode_c1_tps 11.2 tok/s · lab_prefill_tps 258 tok/s gates T21 2026-10-04 16:30 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 load_seconds 54.9 s T21 2026-10-04 16:23 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 ttft_prefill_tps 384 tok/s T20b 2026-10-04 16:14 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 lab_decode_c1_tps 9.10 tok/s · lab_prefill_tps 370 tok/s gates T20b 2026-10-04 16:13 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 load_seconds 48.9 s T20b 2026-10-04 16:00 AEST ✓ pass Flash-Next lab llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 lab_decode_c1_tps 57.1 tok/s · lab_prefill_tps 608 tok/s T11 2026-10-04 15:49 AEST ! fail Flash-Next stability llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 decode_window_min_tps 38.6 tok/s · tool_calls 70 calls T11 2026-10-04 15:45 AEST ✓ pass Flash-Next TTFT llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 ttft_prefill_tps 987 tok/s T11 2026-10-04 15:39 AEST ✓ pass Flash-Next lab llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 lab_decode_c1_tps 56.9 tok/s · lab_prefill_tps 605 tok/s T11 2026-10-04 15:38 AEST ✓ pass Flash-Next load llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 load_seconds 39.9 s T11 2026-10-04 15:19 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 lab_decode_c1_tps 8.30 tok/s · lab_prefill_tps 156 tok/s gates T21 2026-10-04 15:08 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 decode_window_min_tps 7.81 tok/s · tool_calls 25 calls T21 2026-10-04 14:52 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 ttft_prefill_tps 149 tok/s T21 2026-10-04 14:26 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 lab_decode_c1_tps 8.00 tok/s · lab_prefill_tps 139 tok/s gates T21 2026-10-04 14:25 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 load_seconds 91.2 s T21 2026-10-04 14:24 AEST ✕ crash GLM load ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012 load_seconds unavailable load T22c 2026-10-04 14:23 AEST ✕ crash GLM load ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012 load_seconds unavailable load T22c 2026-10-04 14:14 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 288 tok/s gates T20b 2026-10-04 14:04 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 decode_window_min_tps 8.85 tok/s · tool_calls 27 calls T20b 2026-10-04 13:55 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 ttft_prefill_tps 321 tok/s T20b 2026-10-04 13:42 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 288 tok/s gates T20b 2026-10-04 13:41 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 load_seconds 45.9 s T20b 2026-10-04 13:29 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 ttft_prefill_tps 213 tok/s T20b 2026-10-04 13:06 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 201 tok/s gates T20b 2026-10-04 13:05 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 load_seconds 45.9 s T20b 2026-10-04 12:49 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 lab_decode_c1_tps 11.8 tok/s · lab_prefill_tps 218 tok/s gates T21 2026-10-04 12:39 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 decode_window_min_tps 11.0 tok/s · tool_calls 24 calls T21 2026-10-04 12:27 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 ttft_prefill_tps 240 tok/s T21 2026-10-04 12:16 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 lab_decode_c1_tps 11.8 tok/s · lab_prefill_tps 219 tok/s gates T21 2026-10-04 12:15 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 load_seconds 54.9 s T21 2026-10-04 11:59 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 ttft_prefill_tps 159 tok/s T21 2026-10-04 11:39 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 lab_decode_c1_tps 12.2 tok/s · lab_prefill_tps 153 tok/s gates T21 2026-10-04 11:38 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 load_seconds 55.0 s T21 2026-10-04 10:42 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 lab_decode_c1_tps 10.4 tok/s · lab_prefill_tps 132 tok/s gates T21 2026-10-04 10:31 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 decode_window_min_tps 9.86 tok/s · tool_calls 22 calls T21 2026-10-04 10:12 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 ttft_prefill_tps 130 tok/s T21 2026-10-04 09:30 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 lab_decode_c1_tps 10.5 tok/s · lab_prefill_tps 126 tok/s gates T21 2026-10-04 09:29 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 load_seconds 79.7 s T21 2026-10-04 09:17 AEST ✕ crash GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 lab_decode_c1_tps unavailable · lab_prefill_tps unavailable harness T21 2026-10-04 09:07 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 decode_window_min_tps 11.4 tok/s · tool_calls 26 calls T21 2026-10-04 08:51 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 ttft_prefill_tps 166 tok/s T21 2026-10-04 08:27 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 lab_decode_c1_tps 12.3 tok/s · lab_prefill_tps 158 tok/s gates T21 2026-10-04 08:25 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 load_seconds 101 s T21 2026-10-04 08:17 AEST ! fail Flash-Next stability llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 decode_window_min_tps unavailable · tool_calls 0 calls T11 2026-10-04 08:06 AEST ✓ pass Flash-Next load llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 load_seconds 39.9 s T11 2026-10-04 06:02 AEST ✕ crash GLM stability ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 decode_window_min_tps unavailable · tool_calls 0 calls measure T22b 2026-10-04 05:48 AEST ✓ pass GLM TTFT ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 ttft_prefill_tps 199 tok/s T22b 2026-10-04 05:33 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 lab_decode_c1_tps 8.60 tok/s · lab_prefill_tps 147 tok/s gates T22b 2026-10-04 05:32 AEST ✓ pass GLM load ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 load_seconds 51.9 s T22b 2026-10-04 05:16 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 199 tok/s gates T20b 2026-10-04 05:06 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 decode_window_min_tps 8.85 tok/s · tool_calls 30 calls T20b 2026-10-04 04:54 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 ttft_prefill_tps 212 tok/s T20b 2026-10-04 04:26 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 200 tok/s gates T20b 2026-10-04 04:26 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 load_seconds 45.9 s T20b 2026-10-04 03:40 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 lab_decode_c1_tps 8.40 tok/s · lab_prefill_tps 71 tok/s gates T20b 2026-10-04 03:30 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 decode_window_min_tps 7.65 tok/s · tool_calls 25 calls T20b 2026-10-04 02:55 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 ttft_prefill_tps 75.9 tok/s T20b 2026-10-04 02:26 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 lab_decode_c1_tps 8.30 tok/s · lab_prefill_tps 65 tok/s gates T20b 2026-10-04 02:25 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 load_seconds 58.2 s T20b 2026-10-04 01:51 AEST ✕ crash GLM stability ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 decode_window_min_tps unavailable · tool_calls 0 calls measure T22b 2026-10-04 01:37 AEST ✓ pass GLM TTFT ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 ttft_prefill_tps 201 tok/s T22b 2026-10-04 01:19 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 lab_decode_c1_tps 8.20 tok/s · lab_prefill_tps 147 tok/s gates T22b 2026-10-04 01:18 AEST ✓ pass GLM load ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 load_seconds 47.7 s T22b 2026-10-04 01:01 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 lab_decode_c1_tps 9.50 tok/s · lab_prefill_tps 151 tok/s gates T22b 2026-10-04 00:51 AEST ! fail GLM stability ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 decode_window_min_tps 9.46 tok/s · tool_calls 0 calls T22b 2026-10-04 00:37 AEST ✓ pass GLM TTFT ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 ttft_prefill_tps 203 tok/s T22b 2026-10-04 00:22 AEST ! fail GLM lab ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 lab_decode_c1_tps 10.0 tok/s · lab_prefill_tps 151 tok/s gates T22b 2026-10-04 00:22 AEST ✓ pass GLM load ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 load_seconds 46.1 s T22b 2026-10-03 23:31 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 lab_decode_c1_tps 9.30 tok/s · lab_prefill_tps 80 tok/s gates T20b 2026-10-03 23:21 AEST ! fail GLM stability llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 decode_window_min_tps 8.80 tok/s · tool_calls 19 calls T20b 2026-10-03 22:50 AEST ✓ pass GLM TTFT llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 ttft_prefill_tps 84.4 tok/s T20b 2026-10-03 21:59 AEST ! fail GLM lab llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 lab_decode_c1_tps 9.20 tok/s · lab_prefill_tps 78 tok/s gates T20b 2026-10-03 21:58 AEST ✓ pass GLM load llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 load_seconds 49.0 s T20b 2026-10-03 12:48 AEST ! fail GLM lab glm-llamacpp-iq2m-128k-1x-gpu2 lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 81 tok/s gates T20b 2026-10-03 12:38 AEST ! fail GLM stability glm-llamacpp-iq2m-128k-1x-gpu2 decode_window_min_tps 8.91 tok/s · tool_calls 24 calls T20b 2026-10-03 12:07 AEST ✓ pass GLM TTFT glm-llamacpp-iq2m-128k-1x-gpu2 ttft_prefill_tps 86.3 tok/s T20b 2026-10-03 11:41 AEST ! fail GLM lab glm-llamacpp-iq2m-128k-1x-gpu2 lab_decode_c1_tps 9.60 tok/s · lab_prefill_tps 81 tok/s gates T20b 2026-10-03 11:41 AEST ✓ pass GLM load glm-llamacpp-iq2m-128k-1x-gpu2 load_seconds 45.9 s T20b 2026-10-03 11:14 AEST ✓ pass Flash-Next lab fn-llamacpp-iq4xs-128k-1x-fit-gpu2 lab_decode_c1_tps 26.1 tok/s · lab_prefill_tps 157 tok/s T11 2026-10-03 11:03 AEST ! fail Flash-Next stability fn-llamacpp-iq4xs-128k-1x-fit-gpu2 decode_window_min_tps 23.8 tok/s · tool_calls 49 calls T11 2026-10-03 10:49 AEST ✓ pass Flash-Next TTFT fn-llamacpp-iq4xs-128k-1x-fit-gpu2 ttft_prefill_tps 269 tok/s T11 2026-10-03 10:29 AEST ✓ pass Flash-Next lab fn-llamacpp-iq4xs-128k-1x-fit-gpu2 lab_decode_c1_tps 26.5 tok/s · lab_prefill_tps 157 tok/s T11 2026-10-03 10:29 AEST ✓ pass Flash-Next load fn-llamacpp-iq4xs-128k-1x-fit-gpu2 load_seconds 27.8 s T11 2026-10-03 10:28 AEST ✕ crash Flash-Next load fn-llamacpp-iq4xs-128k-1x-gpu2 load_seconds unavailable load T11 2026-10-03 08:17 AEST ✓ pass Flash-Next lab fn-trellis-3.05-0xsero-main-x8-gpu2 lab_decode_c1_tps 55.3 tok/s · lab_prefill_tps 2,228 tok/s T10 2026-10-03 08:06 AEST ! fail Flash-Next stability fn-trellis-3.05-0xsero-main-x8-gpu2 decode_window_min_tps 38.1 tok/s · tool_calls 63 calls T10 2026-10-03 08:06 AEST ✓ pass Flash-Next quality panel fn-trellis-3.05-0xsero-main-x8-gpu2 top1_agreement 0.99234 fraction · mean_kl 0.0009667 nats T10 2026-10-03 08:04 AEST ✓ pass Flash-Next TTFT fn-trellis-3.05-0xsero-main-x8-gpu2 ttft_prefill_tps 2,263 tok/s T10 2026-10-03 07:48 AEST ✓ pass Flash-Next kit sweep fn-trellis-3.05-0xsero-main-x8-gpu2 decode_c1_tps 46.9 tok/s · prefill_32k_tps 2,433 tok/s T10 2026-10-03 07:43 AEST ✓ pass Flash-Next lab fn-trellis-3.05-0xsero-main-x8-gpu2 lab_decode_c1_tps 49.3 tok/s · lab_prefill_tps 2,176 tok/s T10 2026-10-03 07:40 AEST ✓ pass Flash-Next load fn-trellis-3.05-0xsero-main-x8-gpu2 load_seconds 179 s T10 2026-10-03 06:54 AEST ✓ pass none nvme nvme-rand-1tb-ngram rand_read_iops 168,600 IOPS · rand_read_gbps 3.35 GB/s T05a 2026-10-03 06:53 AEST ✓ pass none hw bw hw-bw-gpu0-current h2d_gbps 6.03 GB/s · d2h_gbps 6.60 GB/s T05a 2026-10-03 06:53 AEST ✓ pass none hw bw hw-bw-gpu2-current h2d_gbps 13.5 GB/s · d2h_gbps 13.2 GB/s T05a
Runs / none
20261002T205342Z-hw-bw-gpu2-current-hw_bw ✓ pass hw bw T05a eligibility: none
Config hw-bw-gpu2-current Started 2026-10-03 06:53 AEST Finished 2026-10-03 06:53 AEST (0.1 min) Exit status 0 Notes 1 GiB pinned transfers; full table in detail
Launch no container: static measurement
Metrics context.configured – context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run h2d_gbps 13.5 GB/s {"1MB_pin": 13.15, "1MB_page": 13.02, "4MB_pin": 13.39, "4MB_page": 13.14, "16MB_pin": 13.45, "16MB_page": 13.19, "64MB_pin": 13.46, "64MB_page": 13.18, "256MB_pin": 13.47, "256MB_page": 13.16, "1024MB_pin": 13.47, "1024MB_page": 13.11}d2h_gbps 13.2 GB/s {"1MB_pin": 12.98, "1MB_page": 7.19, "4MB_pin": 13.14, "4MB_page": 10.06, "16MB_pin": 13.2, "16MB_page": 11.35, "64MB_pin": 13.21, "64MB_page": 12.57, "256MB_pin": 13.21, "256MB_page": 12.61, "1024MB_pin": 13.22, "1024MB_page": 12.69}host_memcpy_gbps 16.6 GB/s
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 483c32c7f11d39f2f1e84c87602db3e166dc9fff cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/none/20261002T205342Z-hw-bw-gpu2-current-hw_bw.json
Runs / none
20261002T205350Z-hw-bw-gpu0-current-hw_bw ✓ pass hw bw T05a eligibility: none
Config hw-bw-gpu0-current Started 2026-10-03 06:53 AEST Finished 2026-10-03 06:54 AEST (0.2 min) Exit status 0 Notes 1 GiB pinned transfers; full table in detail
Launch no container: static measurement
Metrics context.configured – context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run h2d_gbps 6.03 GB/s {"1MB_pin": 6.48, "1MB_page": 6.08, "4MB_pin": 6.6, "4MB_page": 6.1, "16MB_pin": 6.57, "16MB_page": 6.11, "64MB_pin": 6.09, "64MB_page": 6.05, "256MB_pin": 6.03, "256MB_page": 6.01, "1024MB_pin": 6.03, "1024MB_page": 5.99}d2h_gbps 6.60 GB/s {"1MB_pin": 6.51, "1MB_page": 4.62, "4MB_pin": 6.58, "4MB_page": 5.88, "16MB_pin": 6.59, "16MB_page": 6.34, "64MB_pin": 6.6, "64MB_page": 6.48, "256MB_pin": 6.6, "256MB_page": 6.51, "1024MB_pin": 6.6, "1024MB_page": 6.52}host_memcpy_gbps 16.6 GB/s
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 483c32c7f11d39f2f1e84c87602db3e166dc9fff cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/none/20261002T205350Z-hw-bw-gpu0-current-hw_bw.json
Runs / none
20261002T205409Z-nvme-rand-1tb-ngram-nvme_rand ✓ pass nvme T05a eligibility: none
Config nvme-rand-1tb-ngram Started 2026-10-03 06:54 AEST Finished 2026-10-03 06:55 AEST (1.1 min) Exit status 0 Notes ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5/ngram_embedding.safetensors
Launch no container: static measurement
Metrics context.configured – context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run rand_read_iops 168,600 IOPS "16 KiB, 32 threads"rand_read_gbps 3.35 GB/s [{"bs": 4096, "threads": 1, "GBps": 0.075, "kIOPS": 18.2}, {"bs": 4096, "threads": 8, "GBps": 0.502, "kIOPS": 122.6}, {"bs": 4096, "threads": 32, "GBps": 0.714, "kIOPS": 174.4}, {"bs": 4096, "threads": 64, "GBps": 0.697, "kIOPS": 170.3}, {"bs": 16384, "threads": 1, "GBps": 0.2, "kIOPS": 12.2}, {"bs": 16384, "threads": 8, "GBps": 1.328, "kIOPS": 81.1}, {"bs": 16384, "threads": 32, "GBps": 2.763, "kIOPS": 168.6}, {"bs": 16384, "threads": 64, "GBps": 2.678, "kIOPS": 163.4}, {"bs": 65536, "threads": 1, "GBps": 0.47, "kIOPS": 7.2}, {"bs": 65536, "threads": 8, "GBps": 2.96, "kIOPS": 45.2}, {"bs": 65536, "threads": 32, "GBps": 3.35, "kIOPS": 51.1}, {"bs": 65536, "threads": 64, "GBps": 3.35, "kIOPS": 51.1}, {"bs": 1048576, "threads": 1, "GBps": 1.972, "kIOPS": 1.9}, {"bs": 1048576, "threads": 8, "GBps": 3.35, "kIOPS": 3.2}, {"bs": 1048576, "threads": 32, "GBps": 3.353, "kIOPS": 3.2}, {"bs": 1048576, "threads": 64, "GBps": 3.355, "kIOPS": 3.2}]
Cache, layout and host Cache warm Layout current · 0 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 483c32c7f11d39f2f1e84c87602db3e166dc9fff cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/none/20261002T205409Z-nvme-rand-1tb-ngram-nvme_rand.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T214031Z-fn-trellis-3.05-0xsero-main-x8-gpu2-load ✓ pass load T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 07:40 AEST Finished 2026-10-03 07:43 AEST (3.0 min) Exit status None Notes –
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 179 s host_ram_drop_gb 63.7 GB vram_ready_mib 21,060 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T214031Z-fn-trellis-3.05-0xsero-main-x8-gpu2-load.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T214330Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab ✓ pass lab T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 07:43 AEST Finished 2026-10-03 07:48 AEST (5.3 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 49.3 tok/s lab_prefill_tps 2,176 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T214330Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T214848Z-fn-trellis-3.05-0xsero-main-x8-gpu2-kit_sweep ✓ pass kit sweep T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 07:48 AEST Finished 2026-10-03 08:04 AEST (15.9 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,292 tok/s {"min": 2277.9, "max": 2305.2, "n": 3}prefill_16k_tps 2,401 tok/s {"min": 2400.6, "max": 2403.1, "n": 3}prefill_32k_tps 2,433 tok/s {"min": 2431.6, "max": 2433.5, "n": 3}prefill_64k_tps 2,415 tok/s {"min": 2413.8, "max": 2415.0, "n": 3}decode_c1_tps 46.9 tok/s {"min": 45.1, "max": 48.7, "power_w": null}decode_c2_tps 51.6 tok/s {"min": 51.1, "max": 52.11, "power_w": null}decode_c3_tps 53.9 tok/s {"min": 51.82, "max": 55.98, "power_w": null}decode_c4_tps 51.8 tok/s {"min": 50.57, "max": 53.12, "power_w": null}decode_c1_32k_tps 48.5 tok/s {"min": 48.51, "max": 48.6, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T214848Z-fn-trellis-3.05-0xsero-main-x8-gpu2-kit_sweep.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T220441Z-fn-trellis-3.05-0xsero-main-x8-gpu2-ttft ✓ pass TTFT T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 08:04 AEST Finished 2026-10-03 08:06 AEST (1.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,263 tok/s {"prompt_tokens": [13679, 13707, 13790], "ttft_s": [6.049, 6.056, 6.045]}ttft_prefill_32k_tps 2,368 tok/s {"prompt_tokens": [54780, 54797, 54620], "ttft_s": [23.132, 23.12, 23.139]}ttft_prefill_tps 2,263 tok/s {"prompt_tokens": [13679, 13707, 13790], "ttft_s": [6.049, 6.056, 6.045]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T220441Z-fn-trellis-3.05-0xsero-main-x8-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T220609Z-fn-trellis-3.05-0xsero-main-x8-gpu2-quality_panel ✓ pass quality panel T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 08:06 AEST Finished 2026-10-03 08:06 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.99234 fraction mean_kl 0.0009667 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T220609Z-fn-trellis-3.05-0xsero-main-x8-gpu2-quality_panel.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T220633Z-fn-trellis-3.05-0xsero-main-x8-gpu2-stability ! fail stability T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 08:06 AEST Finished 2026-10-03 08:17 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 63 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.077 < floor 50.0
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 30,034 tokens context.headroom_min 174,233 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 63 calls {"per_min": 6.3, "requests": 45}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 38.1 tok/s window 1: 42.97 tok/s window 2: 43.5 tok/s window 3: 47.28 tok/s window 4: 41.43 tok/s window 5: 43.59 tok/s window 6: 42.67 tok/s min 41.43 · max 47.28 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 331, "min_at_generation_s": 63.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.1 tok/s {"generation_seconds": 450.481, "generated_tokens": 19411.4}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 208.0, "requests": 18, "tool_calls": 26, "completion_tokens": 6387, "occupied_max": 12502}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 392.0, "requests": 27, "tool_calls": 37, "completion_tokens": 13133, "occupied_max": 30034}] listtasks_over_64k [] instance idscompletion_tokens 19,520 tokens {"finish_reasons": {"tool_calls": 45}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 63 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 38.1 tok/s, whole run 43.1 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T220633Z-fn-trellis-3.05-0xsero-main-x8-gpu2-stability.json
Runs / Qwen3.8-Flash-Next / fn-trellis-3.05-0xsero-main-x8-gpu2
20261002T221703Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab ✓ pass lab T10 eligibility: none
Config fn-trellis-3.05-0xsero-main-x8-gpu2 Started 2026-10-03 08:17 AEST Finished 2026-10-03 08:22 AEST (5.0 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261002T214031Z-lab-fn-trellis-3.05-0xsero-main-x8-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-trellis-3.05-0xsero-main-x8-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.json sha256 248d18e53c5004810964b493e2676bb465a9386a5c527a8b88e67cbfd0dffa68MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.3 tok/s lab_prefill_tps 2,228 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 966c1355dde3774c7406f9d18e6b8468d4a10fdb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261002T221703Z-fn-trellis-3.05-0xsero-main-x8-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-gpu2
20261003T002815Z-fn-llamacpp-iq4xs-128k-1x-gpu2-load ✕ crash load T11 eligibility: none failed stage: load
Config fn-llamacpp-iq4xs-128k-1x-gpu2 Started 2026-10-03 10:28 AEST Finished 2026-10-03 10:28 AEST (0.4 min) Exit status 1 Notes user, abort
0.18.994.031 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 65358.17 MiB on device 0: cudaMalloc failed: out of memory
0.18.994.037 E alloc_tensor_range: failed to allocate CUDA0 buffer of size 68533006336
0.19.139.695 E llama_model_load: error loading model: unable to allocate CUDA0 buffer
0.19.139.702 E llama_model_load_from_file_impl: failed to load model
0.19.139.708 E cmn common_init_: failed to load model '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf'
0.19.139.713 E srv load_model: failed to load model, '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf'
0.19.139.715 I srv operator(): operator(): cleaning up before exit...
0.19.140.570 E srv llama_server: exiting due to model loading error
Launch recorded by tools/run.py (artifacts/runs/20261003T002815Z-lab-fn-llamacpp-iq4xs-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 -ot per_layer_token_embd.weight=CPU Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x.json sha256 697275c75292e4ef45a02a99b2a98c23b7a55630ed0430af0972a1dcf84b47d8MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 -ot per_layer_token_embd.weight=CPUenv CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds unavailable not produced (crash/load) host_ram_drop_gb unavailable not produced (crash/load) vram_ready_mib unavailable not produced (crash/load)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1fa796f9d509705f24dc1870357869cf7c293bc2 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T002815Z-fn-llamacpp-iq4xs-128k-1x-gpu2-load.json
Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2
20261003T002901Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-load ✓ pass load T11 eligibility: none
Config fn-llamacpp-iq4xs-128k-1x-fit-gpu2 Started 2026-10-03 10:29 AEST Finished 2026-10-03 10:29 AEST (0.5 min) Exit status None Notes –
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 27.8 s host_ram_drop_gb 2.30 GB vram_ready_mib 22,684 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T002901Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-load.json
Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2
20261003T002929Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab ✓ pass lab T11 eligibility: none
Config fn-llamacpp-iq4xs-128k-1x-fit-gpu2 Started 2026-10-03 10:29 AEST Finished 2026-10-03 10:49 AEST (19.9 min) Exit status 0 Notes served=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 26.5 tok/s lab_prefill_tps 157 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T002929Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2
20261003T004923Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-ttft ✓ pass TTFT T11 eligibility: none
Config fn-llamacpp-iq4xs-128k-1x-fit-gpu2 Started 2026-10-03 10:49 AEST Finished 2026-10-03 11:03 AEST (14.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 269 tok/s {"prompt_tokens": [13827, 13626, 13781], "ttft_s": [50.985, 51.655, 51.198]}ttft_prefill_32k_tps 230 tok/s {"prompt_tokens": [54692, 54711, 54604], "ttft_s": [237.983, 237.379, 240.167]}ttft_prefill_tps 269 tok/s {"prompt_tokens": [13827, 13626, 13781], "ttft_s": [50.985, 51.655, 51.198]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T004923Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2
20261003T010352Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-stability ! fail stability T11 eligibility: none
Config fn-llamacpp-iq4xs-128k-1x-fit-gpu2 Started 2026-10-03 11:03 AEST Finished 2026-10-03 11:14 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 49 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 23.83 < floor 50.0
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 16,819 tokens context.headroom_min 114,087 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 49 calls {"per_min": 4.9, "requests": 32}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 23.8 tok/s window 1: 24.81 tok/s window 2: 24.06 tok/s window 3: 25.42 tok/s window 4: 24.9 tok/s window 5: 24.6 tok/s min 24.06 · max 25.42 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 271, "min_at_generation_s": 105.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 24.8 tok/s {"generation_seconds": 390.845, "generated_tokens": 9687.8}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 306.9, "requests": 17, "tool_calls": 26, "completion_tokens": 4940, "occupied_max": 16819}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 293.1, "requests": 16, "tool_calls": 24, "completion_tokens": 4788, "occupied_max": 12213}] listtasks_over_64k [] instance idscompletion_tokens 9,728 tokens {"finish_reasons": {"tool_calls": 32}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 49 tool calls, 0 parse failures, 1 repetition hits, tasks solved 1/2, decode min window 23.8 tok/s, whole run 24.8 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T010352Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-stability.json
Runs / Qwen3.8-Flash-Next / fn-llamacpp-iq4xs-128k-1x-fit-gpu2
20261003T011420Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab ✓ pass lab T11 eligibility: none
Config fn-llamacpp-iq4xs-128k-1x-fit-gpu2 Started 2026-10-03 11:14 AEST Finished 2026-10-03 11:40 AEST (26.5 min) Exit status 0 Notes served=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T002901Z-lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2/docker-run.txt) Copy
docker run -d --name lab-fn-llamacpp-iq4xs-128k-1x-fit-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-1x-fit.json sha256 67f1655622a5b08eae6fb8c89ca4a189aab6f7578a1a740baae4832ddc4e8160MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 26.1 tok/s lab_prefill_tps 157 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1fa796f9d509705f24dc1870357869cf7c293bc2 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T011420Z-fn-llamacpp-iq4xs-128k-1x-fit-gpu2-lab.json
Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2
20261003T014103Z-glm-llamacpp-iq2m-128k-1x-gpu2-load ✓ pass load T20b eligibility: none
Config glm-llamacpp-iq2m-128k-1x-gpu2 Started 2026-10-03 11:41 AEST Finished 2026-10-03 11:41 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 45.9 s host_ram_drop_gb 2.23 GB vram_ready_mib 22,784 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 4b12eecb28cf351fc09677fc509debe454394308 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T014103Z-glm-llamacpp-iq2m-128k-1x-gpu2-load.json
Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2
20261003T014149Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config glm-llamacpp-iq2m-128k-1x-gpu2 Started 2026-10-03 11:41 AEST Finished 2026-10-03 12:07 AEST (26.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.60 tok/s lab_prefill_tps 81 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 4b12eecb28cf351fc09677fc509debe454394308 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T014149Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2
20261003T020751Z-glm-llamacpp-iq2m-128k-1x-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config glm-llamacpp-iq2m-128k-1x-gpu2 Started 2026-10-03 12:07 AEST Finished 2026-10-03 12:38 AEST (30.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 86.3 tok/s {"prompt_tokens": [10742, 10816, 10721], "ttft_s": [124.451, 126.526, 123.598]}ttft_prefill_32k_tps 88.5 tok/s {"prompt_tokens": [42736, 42800, 42830], "ttft_s": [485.781, 483.408, 483.895]}ttft_prefill_tps 86.3 tok/s {"prompt_tokens": [10742, 10816, 10721], "ttft_s": [124.451, 126.526, 123.598]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bff973e82a9b895daba523707af9c56454e76a39 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T020751Z-glm-llamacpp-iq2m-128k-1x-gpu2-ttft.json
Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2
20261003T023819Z-glm-llamacpp-iq2m-128k-1x-gpu2-stability ! fail stability T20b eligibility: none
Config glm-llamacpp-iq2m-128k-1x-gpu2 Started 2026-10-03 12:38 AEST Finished 2026-10-03 12:48 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.91 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 9,047 tokens context.headroom_min 121,785 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 24 calls {"per_min": 2.4, "requests": 13}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 8.91 tok/s window 1: 8.98 tok/s window 2: 9.23 tok/s window 3: 9.09 tok/s window 4: 9.18 tok/s window 5: 9.04 tok/s window 6: 9.14 tok/s min 8.98 · max 9.23 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 328, "min_at_generation_s": 90.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 9.14 tok/s {"generation_seconds": 447.593, "generated_tokens": 4091.5}tasks_attempted 1 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 600.0, "requests": 14, "tool_calls": 24, "completion_tokens": 4105, "occupied_max": 9047}] listtasks_over_64k [] instance idscompletion_tokens 4,105 tokens {"finish_reasons": {"tool_calls": 13}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 24 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/1, decode min window 8.91 tok/s, whole run 9.14 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bff973e82a9b895daba523707af9c56454e76a39 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T023819Z-glm-llamacpp-iq2m-128k-1x-gpu2-stability.json
Runs / GLM-5.3-Flash / glm-llamacpp-iq2m-128k-1x-gpu2
20261003T024824Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config glm-llamacpp-iq2m-128k-1x-gpu2 Started 2026-10-03 12:48 AEST Finished 2026-10-03 13:14 AEST (25.9 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T014103Z-lab-glm-llamacpp-iq2m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm-llamacpp-iq2m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x.json sha256 01715887da6d653bd4cc63d959d62744ec8f44f30ca1a60627052180981e3fc1MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.60 tok/s lab_prefill_tps 81 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bff973e82a9b895daba523707af9c56454e76a39 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T024824Z-glm-llamacpp-iq2m-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
20261003T115813Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-load ✓ pass load T20b eligibility: none
Config llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 Started 2026-10-03 21:58 AEST Finished 2026-10-03 21:59 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 49.0 s host_ram_drop_gb 2.22 GB vram_ready_mib 22,626 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 73bf82fd3254285cf46014dbef7e02e6ef6db9b3 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T115813Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
20261003T115902Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 Started 2026-10-03 21:59 AEST Finished 2026-10-03 22:50 AEST (51.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.20 tok/s lab_prefill_tps 78 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bc100ca481d268577db82a7adcd92133db2241e4 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T115902Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
20261003T125002Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 Started 2026-10-03 22:50 AEST Finished 2026-10-03 23:21 AEST (31.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 84.4 tok/s {"prompt_tokens": [10644, 10721, 10688], "ttft_s": [126.044, 127.442, 125.735]}ttft_prefill_32k_tps 84.9 tok/s {"prompt_tokens": [42901, 42759, 42872], "ttft_s": [504.114, 504.375, 505.138]}ttft_prefill_tps 84.4 tok/s {"prompt_tokens": [10644, 10721, 10688], "ttft_s": [126.044, 127.442, 125.735]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bc100ca481d268577db82a7adcd92133db2241e4 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T125002Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
20261003T132135Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-stability ! fail stability T20b eligibility: none
Config llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 Started 2026-10-03 23:21 AEST Finished 2026-10-03 23:31 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 19 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.8 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json
Metrics context.configured 131,072 tokens context.occupied_max 6,658 tokens context.headroom_min 123,658 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 19 calls {"per_min": 1.9, "requests": 11}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 8.80 tok/s window 1: 8.89 tok/s window 2: 8.87 tok/s window 3: 8.93 tok/s window 4: 8.88 tok/s window 5: 9.0 tok/s min 8.87 · max 9 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 267, "min_at_generation_s": 131.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 8.93 tok/s {"generation_seconds": 386.503, "generated_tokens": 3451.5}tasks_attempted 1 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 600.0, "requests": 12, "tool_calls": 20, "completion_tokens": 3463, "occupied_max": 6658}] listtasks_over_64k [] instance idscompletion_tokens 3,463 tokens {"finish_reasons": {"tool_calls": 11}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 19 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/1, decode min window 8.80 tok/s, whole run 8.93 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bc100ca481d268577db82a7adcd92133db2241e4 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T132135Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2
20261003T133139Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 Started 2026-10-03 23:31 AEST Finished 2026-10-04 00:21 AEST (50.3 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T115813Z-lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-Q2_K:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-q2-k-128k-1x.json sha256 f7ab285db47877ab680a678ead1d3de7960b1a54121d89c804a51dc6c2477de0MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-Q2_K.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.30 tok/s lab_prefill_tps 80 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bc100ca481d268577db82a7adcd92133db2241e4 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T133139Z-llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
20261003T142203Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-load ✓ pass load T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 Started 2026-10-04 00:22 AEST Finished 2026-10-04 00:22 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44eMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 46.1 s host_ram_drop_gb 111 GB vram_ready_mib 14,936 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 57320e8fbf776bca99fc63e2de1647d22ef83f06 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T142203Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-load.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
20261003T142250Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab ! fail lab T22b eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 Started 2026-10-04 00:22 AEST Finished 2026-10-04 00:37 AEST (14.7 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44eMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 10.0 tok/s lab_prefill_tps 151 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 57320e8fbf776bca99fc63e2de1647d22ef83f06 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T142250Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
20261003T143733Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-ttft ✓ pass TTFT T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 Started 2026-10-04 00:37 AEST Finished 2026-10-04 00:51 AEST (13.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44eMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 203 tok/s {"prompt_tokens": [10751, 10785, 10716], "ttft_s": [52.615, 53.39, 52.749]}ttft_prefill_32k_tps 197 tok/s {"prompt_tokens": [42739, 42927, 42933], "ttft_s": [217.584, 218.259, 218.191]}ttft_prefill_tps 203 tok/s {"prompt_tokens": [10751, 10785, 10716], "ttft_s": [52.615, 53.39, 52.749]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 57320e8fbf776bca99fc63e2de1647d22ef83f06 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T143733Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-ttft.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
20261003T145106Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-stability ! fail stability T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 Started 2026-10-04 00:51 AEST Finished 2026-10-04 01:01 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 47 tool-call parse failure(s); decode window min 9.458 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44eMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 1,874 tokens context.headroom_min 129,125 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 0 calls {"per_min": 0.0, "requests": 47}parse_failures 47 responses {"wire": {"unparsed_markup": 47}, "harness_format_errors": {"no_tool_call": 47}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 47 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 9.46 tok/s window 1: 9.49 tok/s window 2: 9.55 tok/s window 3: 9.5 tok/s window 4: 9.46 tok/s window 5: 9.55 tok/s min 9.46 · max 9.55 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 277, "min_at_generation_s": 144.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 9.50 tok/s {"generation_seconds": 396.975, "generated_tokens": 3772.8}tasks_attempted 16 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 51.1, "requests": 3, "tool_calls": 0, "completion_tokens": 310, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 40.8, "requests": 3, "tool_calls": 0, "completion_tokens": 227, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 39.7, "requests": 3, "tool_calls": 0, "completion_tokens": 220, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 43.5, "requests": 3, "tool_calls": 0, "completion_tokens": 251, "occupied_max": 1874}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 41.3, "requests": 3, "tool_calls": 0, "completion_tokens": 233, "occupied_max": 1770}, {"attempt": 6, "round": 1, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 34.9, "requests": 3, "tool_calls": 0, "completion_tokens": 174, "occupied_max": 1716}, {"attempt": 7, "round": 1, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 41.1, "requests": 3, "tool_calls": 0, "completion_tokens": 231, "occupied_max": 1641}, {"attempt": 8, "round": 1, "instance_id": "sympy__sympy-20590", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 52.5, "requests": 3, "tool_calls": 0, "completion_tokens": 339, "occupied_max": 1578}, {"attempt": 9, "round": 2, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 31.5, "requests": 3, "tool_calls": 0, "completion_tokens": 230, "occupied_max": 1559}, {"attempt": 10, "round": 2, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 35.7, "requests": 3, "tool_calls": 0, "completion_tokens": 270, "occupied_max": 1641}, {"attempt": 11, "round": 2, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 32.8, "requests": 3, "tool_calls": 0, "completion_tokens": 243, "occupied_max": 1609}, {"attempt": 12, "round": 2, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 33.5, "requests": 3, "tool_calls": 0, "completion_tokens": 249, "occupied_max": 1874}, {"attempt": 13, "round": 2, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 33.7, "requests": 3, "tool_calls": 0, "completion_tokens": 251, "occupied_max": 1770}, {"attempt": 14, "round": 2, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.9, "requests": 3, "tool_calls": 0, "completion_tokens": 178, "occupied_max": 1716}, {"attempt": 15, "round": 2, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 30.0, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1641}, {"attempt": 16, "round": 2, "instance_id": "sympy__sympy-20590", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 32.0, "requests": 3, "tool_calls": 0, "completion_tokens": 198, "occupied_max": 1459}] listtasks_over_64k [] instance idscompletion_tokens 3,821 tokens {"finish_reasons": {"stop": 47}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 0 tool calls, 47 parse failures, 0 repetition hits, tasks solved 0/16, decode min window 9.46 tok/s, whole run 9.50 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 57320e8fbf776bca99fc63e2de1647d22ef83f06 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T145106Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-stability.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2
20261003T150111Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab ! fail lab T22b eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 Started 2026-10-04 01:01 AEST Finished 2026-10-04 01:18 AEST (17.7 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T142203Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x.json sha256 168e8b490c871dcebe6afed1dfd973be9985379eb748e37f17be98b86a86e44eMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 9.50 tok/s lab_prefill_tps 151 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 57320e8fbf776bca99fc63e2de1647d22ef83f06 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T150111Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
20261003T151858Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-load ✓ pass load T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 Started 2026-10-04 01:18 AEST Finished 2026-10-04 01:19 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4MTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 47.7 s host_ram_drop_gb 115 GB vram_ready_mib 22,318 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 664783f004ff58b6fcabb0691d6b1c8298e2e818 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T151858Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-load.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
20261003T151947Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-lab ! fail lab T22b eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 Started 2026-10-04 01:19 AEST Finished 2026-10-04 01:37 AEST (17.7 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4MTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 8.20 tok/s lab_prefill_tps 147 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps unavailable not produced (fail/gates) mtp_acceptance_rate unavailable not produced (fail/gates)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 937eb11054a0e7abd1b0e970f25ea2fda3f4cff9 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T151947Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
20261003T153729Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-ttft ✓ pass TTFT T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 Started 2026-10-04 01:37 AEST Finished 2026-10-04 01:51 AEST (13.9 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4MTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 201 tok/s {"prompt_tokens": [10740, 10677, 10728], "ttft_s": [53.545, 52.784, 53.644]}ttft_prefill_32k_tps 190 tok/s {"prompt_tokens": [42815, 42706, 42534], "ttft_s": [223.459, 224.655, 228.199]}ttft_prefill_tps 201 tok/s {"prompt_tokens": [10740, 10677, 10728], "ttft_s": [53.545, 52.784, 53.644]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 937eb11054a0e7abd1b0e970f25ea2fda3f4cff9 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T153729Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-ttft.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2
20261003T155126Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-stability ✕ crash stability T22b eligibility: none failed stage: measure
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 Started 2026-10-04 01:51 AEST Finished 2026-10-04 02:01 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T151858Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp.json sha256 c2df636d0f6ab990abe7dd3c658b064f3c8df59d819b7fe907ce82a12b84a9c4MTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 1,641 tokens context.headroom_min 129,356 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 0 calls {"per_min": 0.0, "requests": 9}parse_failures 9 responses {"wire": {"unparsed_markup": 9}, "harness_format_errors": {"no_tool_call": 9}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 9 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps unavailable only 57.72 s of generation; need > 120 s for one window decode_whole_run_tps 12.6 tok/s {"generation_seconds": 57.72, "generated_tokens": 729.8}tasks_attempted 5 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom yes {"upstream_connection_failures": 54, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 47.1, "requests": 3, "tool_calls": 0, "completion_tokens": 274, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 36.4, "requests": 3, "tool_calls": 0, "completion_tokens": 248, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 33.8, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 343.5}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 139.3}] listtasks_over_64k [] instance idscompletion_tokens 739 tokens {"finish_reasons": {"stop": 9}}mtp_accepted_tps unavailable not produced (crash/measure) mtp_acceptance_rate unavailable not produced (crash/measure)
Agentic summary: 600 s, 0 tool calls, 9 parse failures, 0 repetition hits, tasks solved 0/5, decode min window – tok/s, whole run 12.6 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 937eb11054a0e7abd1b0e970f25ea2fda3f4cff9 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T155126Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
20261003T162532Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-load ✓ pass load T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 Started 2026-10-04 02:25 AEST Finished 2026-10-04 02:26 AEST (1.0 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 58.2 s host_ram_drop_gb 1.89 GB vram_ready_mib 22,596 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9fc82ec3597413063d86252387463dee533769ed cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T162532Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
20261003T162631Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 Started 2026-10-04 02:26 AEST Finished 2026-10-04 02:55 AEST (28.7 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 8.30 tok/s lab_prefill_tps 65 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9fc82ec3597413063d86252387463dee533769ed cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T162631Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
20261003T165514Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 Started 2026-10-04 02:55 AEST Finished 2026-10-04 03:30 AEST (34.8 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 75.9 tok/s {"prompt_tokens": [10736, 10754, 10667], "ttft_s": [140.833, 141.619, 141.761]}ttft_prefill_32k_tps 77.3 tok/s {"prompt_tokens": [42840, 42803, 42831], "ttft_s": [554.48, 553.698, 555.603]}ttft_prefill_tps 75.9 tok/s {"prompt_tokens": [10736, 10754, 10667], "ttft_s": [140.833, 141.619, 141.761]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9fc82ec3597413063d86252387463dee533769ed cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T165514Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
20261003T173003Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-stability ! fail stability T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 Started 2026-10-04 03:30 AEST Finished 2026-10-04 03:40 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.646 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max 4,875 tokens context.headroom_min 126,123 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 25 calls {"per_min": 2.5, "requests": 15}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 7.65 tok/s window 1: 8.08 tok/s window 2: 7.78 tok/s window 3: 7.99 tok/s window 4: 8.16 tok/s window 5: 8.19 tok/s min 7.78 · max 8.19 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 269, "min_at_generation_s": 156.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 8.08 tok/s {"generation_seconds": 388.925, "generated_tokens": 3143.9}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 377.0, "requests": 10, "tool_calls": 17, "completion_tokens": 2133, "occupied_max": 4875}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 223.0, "requests": 6, "tool_calls": 9, "completion_tokens": 1027, "occupied_max": 4558}] listtasks_over_64k [] instance idscompletion_tokens 3,160 tokens {"finish_reasons": {"tool_calls": 15}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 25 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 7.65 tok/s, whole run 8.08 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9fc82ec3597413063d86252387463dee533769ed cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T173003Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2
20261003T174033Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 Started 2026-10-04 03:40 AEST Finished 2026-10-04 04:25 AEST (45.4 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T162532Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x.json sha256 76879bccb4223161305f6a73f10b3a4f79db3609c18c993495ef170aa8ae9db8MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 8.40 tok/s lab_prefill_tps 71 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9fc82ec3597413063d86252387463dee533769ed cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T174033Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
20261003T182602Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-load ✓ pass load T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 Started 2026-10-04 04:26 AEST Finished 2026-10-04 04:26 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94feMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 45.9 s host_ram_drop_gb 3.54 GB vram_ready_mib 21,944 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d4f093c724ebfd683feed823be35a11d4c009375 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T182602Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
20261003T182648Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 Started 2026-10-04 04:26 AEST Finished 2026-10-04 04:54 AEST (27.5 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94feMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.50 tok/s lab_prefill_tps 200 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d4f093c724ebfd683feed823be35a11d4c009375 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T182648Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
20261003T185415Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 Started 2026-10-04 04:54 AEST Finished 2026-10-04 05:06 AEST (12.2 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94feMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 212 tok/s {"prompt_tokens": [10787, 10703, 10755], "ttft_s": [50.583, 51.91, 50.672]}ttft_prefill_32k_tps 221 tok/s {"prompt_tokens": [42782, 42887, 42759], "ttft_s": [192.559, 193.833, 193.907]}ttft_prefill_tps 212 tok/s {"prompt_tokens": [10787, 10703, 10755], "ttft_s": [50.583, 51.91, 50.672]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d4f093c724ebfd683feed823be35a11d4c009375 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T185415Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
20261003T190629Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-stability ! fail stability T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 Started 2026-10-04 05:06 AEST Finished 2026-10-04 05:16 AEST (10.4 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 30 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.854 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94feMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 9,732 tokens context.headroom_min 121,298 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 30 calls {"per_min": 3.0, "requests": 21}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 8.85 tok/s window 1: 9.19 tok/s window 2: 9.0 tok/s window 3: 8.95 tok/s window 4: 8.92 tok/s window 5: 9.14 tok/s min 8.92 · max 9.19 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 274, "min_at_generation_s": 232.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 9.09 tok/s {"generation_seconds": 393.673, "generated_tokens": 3578.8}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 578.9, "requests": 21, "tool_calls": 30, "completion_tokens": 3601, "occupied_max": 9732}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 21.1, "requests": 1, "tool_calls": 1, "completion_tokens": 0, "occupied_max": 0}] listtasks_over_64k [] instance idscompletion_tokens 3,601 tokens {"finish_reasons": {"tool_calls": 21}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 30 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 8.85 tok/s, whole run 9.09 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d4f093c724ebfd683feed823be35a11d4c009375 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T190629Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2
20261003T191656Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 Started 2026-10-04 05:16 AEST Finished 2026-10-04 05:31 AEST (15.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T182602Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048.json sha256 89ee35a7fb8e6ec8fd63aa7e9e07a5ed1c3448eed3d10cebf357887e810a94feMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 16 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.50 tok/s lab_prefill_tps 199 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d4f093c724ebfd683feed823be35a11d4c009375 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T191656Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
20261003T193228Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-load ✓ pass load T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 Started 2026-10-04 05:32 AEST Finished 2026-10-04 05:33 AEST (0.9 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13aMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 51.9 s host_ram_drop_gb 115 GB vram_ready_mib 22,318 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 956afbaae18f13c44809e1d61a0a1568dca9db78 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T193228Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-load.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
20261003T193321Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-lab ! fail lab T22b eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 Started 2026-10-04 05:33 AEST Finished 2026-10-04 05:48 AEST (15.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13aMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 8.60 tok/s lab_prefill_tps 147 tok/s "context-gate prompt tokens / total request seconds"mtp_acceptance_rate 0.5403 fraction {"draft_tokens": 2889, "accepted_tokens": 1561, "source": "server log delta"}mtp_accepted_tps 8.60 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 956afbaae18f13c44809e1d61a0a1568dca9db78 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T193321Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
20261003T194821Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-ttft ✓ pass TTFT T22b eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 Started 2026-10-04 05:48 AEST Finished 2026-10-04 06:02 AEST (13.9 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13aMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 199 tok/s {"prompt_tokens": [10721, 10696, 10751], "ttft_s": [53.479, 53.836, 54.249]}ttft_prefill_32k_tps 191 tok/s {"prompt_tokens": [42813, 42832, 42764], "ttft_s": [223.74, 224.908, 224.251]}ttft_prefill_tps 199 tok/s {"prompt_tokens": [10721, 10696, 10751], "ttft_s": [53.479, 53.836, 54.249]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 956afbaae18f13c44809e1d61a0a1568dca9db78 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T194821Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-ttft.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2
20261003T200216Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-stability ✕ crash stability T22b eligibility: none failed stage: measure
Config ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 Started 2026-10-04 06:02 AEST Finished 2026-10-04 06:12 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261003T193228Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4.json sha256 aa948a723db410fdb552afee2e3ac6e4d2fa58a7a7d510ddd629de6bd166b13aMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --cpu-moe --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 16 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --ctx-checkpoints 4env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 1,641 tokens context.headroom_min 129,356 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 0 calls {"per_min": 0.0, "requests": 9}parse_failures 9 responses {"wire": {"unparsed_markup": 9}, "harness_format_errors": {"no_tool_call": 9}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 9 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps unavailable only 58.214 s of generation; need > 120 s for one window decode_whole_run_tps 12.5 tok/s {"generation_seconds": 58.214, "generated_tokens": 729.8}tasks_attempted 5 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom yes {"upstream_connection_failures": 54, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 41.0, "requests": 3, "tool_calls": 0, "completion_tokens": 274, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 36.8, "requests": 3, "tool_calls": 0, "completion_tokens": 248, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 34.6, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 341.4}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 146.2}] listtasks_over_64k [] instance idscompletion_tokens 739 tokens {"finish_reasons": {"stop": 9}}mtp_accepted_tps unavailable not produced (crash/measure) mtp_acceptance_rate unavailable not produced (crash/measure)
Agentic summary: 600 s, 0 tool calls, 9 parse failures, 0 repetition hits, tasks solved 0/5, decode min window – tok/s, whole run 12.5 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-31 cpus_offline none machine_checks_this_boot 0
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 956afbaae18f13c44809e1d61a0a1568dca9db78 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T200216Z-ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2-stability.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012
20261003T220620Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-load ✓ pass load T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 Started 2026-10-04 08:06 AEST Finished 2026-10-04 08:07 AEST (0.7 min) Exit status None Notes –
Other runs of this config: load stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T220620Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit.json sha256 f7129eb274fa7a838d7a8928e9e72cec61e157259a237393b1f99b1945e722abMTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layerenv CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 39.9 s host_ram_drop_gb 3.35 GB vram_ready_mib 68,976 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2e517d99a59b71bbf630b44ce6204f3b85717adb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T220620Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-load.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012
20261003T221725Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-stability ! fail stability T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 Started 2026-10-04 08:17 AEST Finished 2026-10-04 08:19 AEST (1.9 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode floor not verifiable (too little generation); agent driver exited 1
Other runs of this config: load stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T220620Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit.json sha256 f7129eb274fa7a838d7a8928e9e72cec61e157259a237393b1f99b1945e722abMTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layerenv CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no successful request context.headroom_min unavailable configured context unknown (set CONFIGURED_CTX) duration_s unavailable driver produced no timing tool_calls 0 calls {"per_min": null, "requests": 0}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": null, "saved": "artifacts: proxy/repetition/"}decode_window_min_tps unavailable only 0.0 s of generation; need > 120 s for one window decode_whole_run_tps unavailable no generation recorded tasks_attempted 0 attempts {"unfinished_at_cap": 0, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved skipped no attempts crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [] listtasks_over_64k [] instance idscompletion_tokens 0 tokens {"finish_reasons": {}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: – s, 0 tool calls, 0 parse failures, 0 repetition hits, tasks solved –/0, decode min window – tok/s, whole run – tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d9944c1c5f811326cc98ee2701073db1981ab9b7 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261003T221725Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
20261003T222524Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-load ✓ pass load T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 Started 2026-10-04 08:25 AEST Finished 2026-10-04 08:27 AEST (1.7 min) Exit status None Notes –
Other runs of this config: load lab ! lab ✕ TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ceMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 101 s host_ram_drop_gb 40.4 GB vram_ready_mib 68,496 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit cf7ad504f20ebd9fc05fbafb689f4809d92ed210 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T222524Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
20261003T222706Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 Started 2026-10-04 08:27 AEST Finished 2026-10-04 08:51 AEST (24.6 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ✕ TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ceMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 12.3 tok/s lab_prefill_tps 158 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit cf7ad504f20ebd9fc05fbafb689f4809d92ed210 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T222706Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
20261003T225145Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-ttft ✓ pass TTFT T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 Started 2026-10-04 08:51 AEST Finished 2026-10-04 09:07 AEST (15.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ✕ TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ceMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 166 tok/s {"prompt_tokens": [10720, 10806, 10675], "ttft_s": [64.46, 64.694, 64.976]}ttft_prefill_32k_tps 175 tok/s {"prompt_tokens": [42882, 42861, 42885], "ttft_s": [244.773, 246.002, 245.262]}ttft_prefill_tps 166 tok/s {"prompt_tokens": [10720, 10806, 10675], "ttft_s": [64.46, 64.694, 64.976]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit cf7ad504f20ebd9fc05fbafb689f4809d92ed210 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T225145Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
20261003T230715Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-stability ! fail stability T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 Started 2026-10-04 09:07 AEST Finished 2026-10-04 09:17 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 26 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.388 < floor 15.0
Other runs of this config: load lab ! lab ✕ TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ceMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 8,118 tokens context.headroom_min 122,865 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 26 calls {"per_min": 2.6, "requests": 15}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 11.4 tok/s window 1: 11.57 tok/s window 2: 11.47 tok/s window 3: 11.64 tok/s min 11.47 · max 11.64 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 143, "min_at_generation_s": 109.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 11.7 tok/s {"generation_seconds": 262.271, "generated_tokens": 3060.1}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 364.3, "requests": 12, "tool_calls": 20, "completion_tokens": 2740, "occupied_max": 8118}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 235.7, "requests": 4, "tool_calls": 6, "completion_tokens": 336, "occupied_max": 2784}] listtasks_over_64k [] instance idscompletion_tokens 3,076 tokens {"finish_reasons": {"tool_calls": 15}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 26 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 11.4 tok/s, whole run 11.7 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit cf7ad504f20ebd9fc05fbafb689f4809d92ed210 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T230715Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012
20261003T231744Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab ✕ crash lab T21 eligibility: none failed stage: harness
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 Started 2026-10-04 09:17 AEST Finished 2026-10-04 09:29 AEST (11.3 min) Exit status 1 Notes –
Other runs of this config: load lab ! lab ✕ TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T222524Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048.json sha256 ada3d8f510a24f0d5811d7af83862f218e4475f6cd2935003b2162f2c40e76ceMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed unavailable not produced (crash/harness) lab_decode_c1_tps unavailable not produced (crash/harness) lab_prefill_tps unavailable not produced (crash/harness) mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit cf7ad504f20ebd9fc05fbafb689f4809d92ed210 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T231744Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
20261003T232915Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-load ✓ pass load T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 Started 2026-10-04 09:29 AEST Finished 2026-10-04 09:30 AEST (1.3 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7afMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 79.7 s host_ram_drop_gb 2.99 GB vram_ready_mib 67,148 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d6382f80d3a94ab1a97311c8a27427f2b5c7f9cb cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T232915Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
20261003T233035Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 Started 2026-10-04 09:30 AEST Finished 2026-10-04 10:12 AEST (42.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7afMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 10.5 tok/s lab_prefill_tps 126 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 94e598f979af99cb8e62f9dacf2a0f1b258ccf3e cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261003T233035Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
20261004T001236Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-ttft ✓ pass TTFT T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 Started 2026-10-04 10:12 AEST Finished 2026-10-04 10:31 AEST (19.3 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7afMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 130 tok/s {"prompt_tokens": [10633, 10740, 10676], "ttft_s": [80.845, 98.056, 82.319]}ttft_prefill_32k_tps 143 tok/s {"prompt_tokens": [42717, 42962, 42901], "ttft_s": [298.369, 301.101, 295.768]}ttft_prefill_tps 130 tok/s {"prompt_tokens": [10633, 10740, 10676], "ttft_s": [80.845, 98.056, 82.319]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 376b9064b928086c6007ed16ab0d9267e111dd92 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T001236Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
20261004T003153Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-stability ! fail stability T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 Started 2026-10-04 10:31 AEST Finished 2026-10-04 10:42 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 22 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 9.855 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7afMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max 5,309 tokens context.headroom_min 125,652 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 22 calls {"per_min": 2.2, "requests": 16}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 9.86 tok/s window 1: 10.15 tok/s window 2: 10.0 tok/s window 3: 10.01 tok/s min 10 · max 10.15 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 148, "min_at_generation_s": 195.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 10.1 tok/s {"generation_seconds": 267.587, "generated_tokens": 2697.7}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 225.2, "requests": 10, "tool_calls": 15, "completion_tokens": 1117, "occupied_max": 3970}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 374.8, "requests": 7, "tool_calls": 7, "completion_tokens": 1598, "occupied_max": 5309}] listtasks_over_64k [] instance idscompletion_tokens 2,715 tokens {"finish_reasons": {"tool_calls": 16}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 22 tool calls, 0 parse failures, 1 repetition hits, tasks solved 1/2, decode min window 9.86 tok/s, whole run 10.1 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 376b9064b928086c6007ed16ab0d9267e111dd92 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T003153Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012
20261004T004223Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 Started 2026-10-04 10:42 AEST Finished 2026-10-04 11:09 AEST (27.2 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261003T232915Z-lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ3_XXS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048.json sha256 4112d34e61cb2bb3861a59a7bf7cc1ce23cbba5e7f1b7cb3ce73f36dcfa2b7afMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ3_XXS.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 10.4 tok/s lab_prefill_tps 132 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 376b9064b928086c6007ed16ab0d9267e111dd92 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T004223Z-llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012
20261004T013855Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-load ✓ pass load T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 Started 2026-10-04 11:38 AEST Finished 2026-10-04 11:39 AEST (0.9 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T013855Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048.json sha256 d46ba43c1169b3fd2e5d00ace10dc78cf7122918e1c259debec506f8a0766d22MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 55.0 s host_ram_drop_gb 3.83 GB vram_ready_mib 65,124 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 640a2b5606870f5adba5d1a5f8a557a075bbdecc dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T013855Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012
20261004T013950Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 Started 2026-10-04 11:39 AEST Finished 2026-10-04 11:59 AEST (19.8 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T013855Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048.json sha256 d46ba43c1169b3fd2e5d00ace10dc78cf7122918e1c259debec506f8a0766d22MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 12.2 tok/s lab_prefill_tps 153 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 640a2b5606870f5adba5d1a5f8a557a075bbdecc dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T013950Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012
20261004T015940Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-ttft ✓ pass TTFT T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 Started 2026-10-04 11:59 AEST Finished 2026-10-04 12:15 AEST (16.0 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T013855Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048.json sha256 d46ba43c1169b3fd2e5d00ace10dc78cf7122918e1c259debec506f8a0766d22MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 159 tok/s {"prompt_tokens": [10694, 10665, 10720], "ttft_s": [67.037, 66.91, 67.278]}ttft_prefill_32k_tps 169 tok/s {"prompt_tokens": [42746, 42816, 42870], "ttft_s": [254.159, 253.337, 253.851]}ttft_prefill_tps 159 tok/s {"prompt_tokens": [10694, 10665, 10720], "ttft_s": [67.037, 66.91, 67.278]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 640a2b5606870f5adba5d1a5f8a557a075bbdecc dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T015940Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T021546Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-load ✓ pass load T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 12:15 AEST Finished 2026-10-04 12:16 AEST (0.9 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 54.9 s host_ram_drop_gb 4.03 GB vram_ready_mib 65,124 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd89b522703b1628ea955d1581e800faae3e8e5f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T021546Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T021642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 12:16 AEST Finished 2026-10-04 12:27 AEST (11.2 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 11.8 tok/s lab_prefill_tps 219 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd89b522703b1628ea955d1581e800faae3e8e5f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T021642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T022756Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-ttft ✓ pass TTFT T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 12:27 AEST Finished 2026-10-04 12:39 AEST (11.1 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 240 tok/s {"prompt_tokens": [10656, 10730, 10756], "ttft_s": [44.441, 45.48, 44.34]}ttft_prefill_32k_tps 242 tok/s {"prompt_tokens": [42911, 42857, 42784], "ttft_s": [177.67, 176.719, 177.974]}ttft_prefill_tps 240 tok/s {"prompt_tokens": [10656, 10730, 10756], "ttft_s": [44.441, 45.48, 44.34]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd89b522703b1628ea955d1581e800faae3e8e5f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T022756Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T023904Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-stability ! fail stability T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 12:39 AEST Finished 2026-10-04 12:49 AEST (10.4 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.035 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 6,174 tokens context.headroom_min 124,753 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 24 calls {"per_min": 2.4, "requests": 16}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 11.0 tok/s window 1: 11.25 tok/s window 2: 11.07 tok/s min 11.07 · max 11.25 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 78, "min_at_generation_s": 109.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 11.3 tok/s {"generation_seconds": 197.184, "generated_tokens": 2222.9}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 266.1, "requests": 13, "tool_calls": 19, "completion_tokens": 2034, "occupied_max": 6174}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 333.9, "requests": 4, "tool_calls": 5, "completion_tokens": 206, "occupied_max": 3300}] listtasks_over_64k [] instance idscompletion_tokens 2,240 tokens {"finish_reasons": {"tool_calls": 16}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 24 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 11.0 tok/s, whole run 11.3 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd89b522703b1628ea955d1581e800faae3e8e5f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T023904Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T024930Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 12:49 AEST Finished 2026-10-04 13:05 AEST (16.4 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T021546Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210.json sha256 1f7fcd2dbb10fca06661e2443bf4355bb609fd7677b4665a7891f97c2f975aa9MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 11.8 tok/s lab_prefill_tps 218 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd89b522703b1628ea955d1581e800faae3e8e5f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T024930Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2
20261004T030555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-load ✓ pass load T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 Started 2026-10-04 13:05 AEST Finished 2026-10-04 13:06 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T030555Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15.json sha256 72b9b0b1ecb00a8819e5a83e80deef181225276cbaa232ea9d7ed944c7391289MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 45.9 s host_ram_drop_gb 3.52 GB vram_ready_mib 21,944 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 0ace6784b9e48161d6b2ab80af2c1887ace48181 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T030555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2
20261004T030642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 Started 2026-10-04 13:06 AEST Finished 2026-10-04 13:29 AEST (22.3 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T030555Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15.json sha256 72b9b0b1ecb00a8819e5a83e80deef181225276cbaa232ea9d7ed944c7391289MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.50 tok/s lab_prefill_tps 201 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bc3d891210f23ffb6504b865b4216f82e08a42ef dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T030642Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2
20261004T032900Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 Started 2026-10-04 13:29 AEST Finished 2026-10-04 13:41 AEST (12.2 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T030555Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15.json sha256 72b9b0b1ecb00a8819e5a83e80deef181225276cbaa232ea9d7ed944c7391289MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 213 tok/s {"prompt_tokens": [10652, 10738, 10639], "ttft_s": [50.031, 50.271, 51.458]}ttft_prefill_32k_tps 222 tok/s {"prompt_tokens": [42844, 42844, 42698], "ttft_s": [192.579, 195.095, 192.023]}ttft_prefill_tps 213 tok/s {"prompt_tokens": [10652, 10738, 10639], "ttft_s": [50.031, 50.271, 51.458]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bc3d891210f23ffb6504b865b4216f82e08a42ef dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T032900Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
20261004T034114Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-load ✓ pass load T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 Started 2026-10-04 13:41 AEST Finished 2026-10-04 13:42 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972beMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 45.9 s host_ram_drop_gb 4.55 GB vram_ready_mib 22,914 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2dbf31a09288732fa781855672b4d25178eb4682 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T034114Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
20261004T034201Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 Started 2026-10-04 13:42 AEST Finished 2026-10-04 13:55 AEST (13.9 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972beMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.60 tok/s lab_prefill_tps 288 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2dbf31a09288732fa781855672b4d25178eb4682 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T034201Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
20261004T035555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 Started 2026-10-04 13:55 AEST Finished 2026-10-04 14:04 AEST (8.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972beMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 321 tok/s {"prompt_tokens": [10699, 10766, 10755], "ttft_s": [33.318, 33.541, 34.552]}ttft_prefill_32k_tps 316 tok/s {"prompt_tokens": [42816, 42866, 42738], "ttft_s": [135.262, 135.596, 135.444]}ttft_prefill_tps 321 tok/s {"prompt_tokens": [10699, 10766, 10755], "ttft_s": [33.318, 33.541, 34.552]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2dbf31a09288732fa781855672b4d25178eb4682 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T035555Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
20261004T040423Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-stability ! fail stability T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 Started 2026-10-04 14:04 AEST Finished 2026-10-04 14:14 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 27 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 8.847 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972beMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 6,632 tokens context.headroom_min 124,337 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 27 calls {"per_min": 2.7, "requests": 17}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 8.85 tok/s window 1: 9.1 tok/s window 2: 8.89 tok/s window 3: 9.14 tok/s window 4: 9.3 tok/s window 5: 9.15 tok/s window 6: 9.01 tok/s min 8.89 · max 9.3 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 314, "min_at_generation_s": 123.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 9.10 tok/s {"generation_seconds": 433.213, "generated_tokens": 3940.0}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 331.4, "requests": 11, "tool_calls": 18, "completion_tokens": 2064, "occupied_max": 6088}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 268.6, "requests": 7, "tool_calls": 9, "completion_tokens": 1894, "occupied_max": 6632}] listtasks_over_64k [] instance idscompletion_tokens 3,958 tokens {"finish_reasons": {"tool_calls": 17}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 27 tool calls, 0 parse failures, 1 repetition hits, tasks solved 1/2, decode min window 8.85 tok/s, whole run 9.10 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2dbf31a09288732fa781855672b4d25178eb4682 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T040423Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2
20261004T041451Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 Started 2026-10-04 14:14 AEST Finished 2026-10-04 14:23 AEST (8.6 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T034114Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096.json sha256 ee2f2a2da672b25768c3764ea94939d349c3e8c3616b4ebf5a18302b922972beMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.60 tok/s lab_prefill_tps 288 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2dbf31a09288732fa781855672b4d25178eb4682 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T041451Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012
20261004T042329Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012-load ✕ crash load T22c eligibility: none failed stage: load
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012 Started 2026-10-04 14:23 AEST Finished 2026-10-04 14:24 AEST (0.7 min) Exit status 1 Notes che_init: CUDA2 KV buffer size = 570.81 MiB
llama_init_from_model: KV self size = 748.00 MiB, c^KV (q8_0): 748.00 MiB, kv^T: not used
llama_init_from_model: KV self indexer size = 704.00 MiB (f16)
llama_init_from_model: CUDA_Host output buffer size = 0.59 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696
llama_init_from_model: failed to allocate compute buffers
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'
Launch recorded by tools/run.py (artifacts/runs/20261004T042329Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210.json sha256 b1d4a0440963d486728825e444f7ac6736f7d39444e483cdf843d738cf41f923MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds unavailable not produced (crash/load) host_ram_drop_gb unavailable not produced (crash/load) vram_ready_mib unavailable not produced (crash/load)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 562ad23a6a0d69a48c21a7611e7c2d9f0984a1ed dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T042329Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012-load.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012
20261004T042414Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012-load ✕ crash load T22c eligibility: none failed stage: load
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012 Started 2026-10-04 14:24 AEST Finished 2026-10-04 14:25 AEST (0.8 min) Exit status 1 Notes che_init: CUDA2 KV buffer size = 570.81 MiB
llama_init_from_model: KV self size = 816.00 MiB, c^KV (q8_0): 816.00 MiB, kv^T: not used
llama_init_from_model: KV self indexer size = 768.00 MiB (f16)
llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696
llama_init_from_model: failed to allocate compute buffers
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'
Launch recorded by tools/run.py (artifacts/runs/20261004T042414Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210.json sha256 9f2b351bbcb5a9f58f9c86180a6023d2ce919c2149f9640d3778c925a0ec8881MTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds unavailable not produced (crash/load) host_ram_drop_gb unavailable not produced (crash/load) vram_ready_mib unavailable not produced (crash/load)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 0c585c1d628c0c4a04e8994c51e5954b718fabea dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T042414Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T042504Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-load ✓ pass load T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 14:25 AEST Finished 2026-10-04 14:26 AEST (1.5 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 91.2 s host_ram_drop_gb 3.05 GB vram_ready_mib 63,084 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T042504Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T042635Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 14:26 AEST Finished 2026-10-04 14:52 AEST (25.7 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 8.00 tok/s lab_prefill_tps 139 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T042635Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T045220Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-ttft ✓ pass TTFT T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 14:52 AEST Finished 2026-10-04 15:08 AEST (16.7 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 149 tok/s {"prompt_tokens": [10777, 10733, 10753], "ttft_s": [130.896, 71.944, 64.716]}ttft_prefill_32k_tps 176 tok/s {"prompt_tokens": [42666, 42838, 42755], "ttft_s": [243.017, 245.943, 242.684]}ttft_prefill_tps 149 tok/s {"prompt_tokens": [10777, 10733, 10753], "ttft_s": [130.896, 71.944, 64.716]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T045220Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T050859Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-stability ! fail stability T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 15:08 AEST Finished 2026-10-04 15:19 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.807 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 6,852 tokens context.headroom_min 123,939 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 25 calls {"per_min": 2.5, "requests": 19}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 7.81 tok/s window 1: 7.91 tok/s window 2: 7.96 tok/s window 3: 8.06 tok/s min 7.91 · max 8.06 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 175, "min_at_generation_s": 107.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 7.95 tok/s {"generation_seconds": 294.515, "generated_tokens": 2340.5}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 277.5, "requests": 9, "tool_calls": 12, "completion_tokens": 1323, "occupied_max": 4597}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 322.5, "requests": 11, "tool_calls": 13, "completion_tokens": 1038, "occupied_max": 6852}] listtasks_over_64k [] instance idscompletion_tokens 2,361 tokens {"finish_reasons": {"tool_calls": 19}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 25 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 7.81 tok/s, whole run 7.95 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T050859Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-stability.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012
20261004T051928Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 Started 2026-10-04 15:19 AEST Finished 2026-10-04 15:38 AEST (19.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T042504Z-lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210- --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210.json sha256 6a17b74e230df72d3cf6d97693f004f278ead699c76a664b7f92b5fe36813807MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 2048 --ubatch-size 2048 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 8.30 tok/s lab_prefill_tps 156 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 743ba9ddf00e0b7eb28b9180241f4a3fb9322a06 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T051928Z-llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012-lab.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
20261004T053844Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-load ✓ pass load T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Started 2026-10-04 15:38 AEST Finished 2026-10-04 15:39 AEST (0.7 min) Exit status None Notes –
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 39.9 s host_ram_drop_gb 2.65 GB vram_ready_mib 65,510 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd8fc15971beec0df168ae2ee8c6c01aae2a7453 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T053844Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-load.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
20261004T053925Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab ✓ pass lab T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Started 2026-10-04 15:39 AEST Finished 2026-10-04 15:45 AEST (6.3 min) Exit status 0 Notes served=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 56.9 tok/s lab_prefill_tps 605 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e17f05543581ac460774a08531a895566f2d209d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T053925Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
20261004T054545Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-ttft ✓ pass TTFT T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Started 2026-10-04 15:45 AEST Finished 2026-10-04 15:49 AEST (3.9 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 987 tok/s {"prompt_tokens": [13709, 13794, 13718], "ttft_s": [14.16, 13.87, 13.896]}ttft_prefill_32k_tps 852 tok/s {"prompt_tokens": [54701, 54603, 54830], "ttft_s": [64.015, 64.121, 64.836]}ttft_prefill_tps 987 tok/s {"prompt_tokens": [13709, 13794, 13718], "ttft_s": [14.16, 13.87, 13.896]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e17f05543581ac460774a08531a895566f2d209d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T054545Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-ttft.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
20261004T054940Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-stability ! fail stability T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Started 2026-10-04 15:49 AEST Finished 2026-10-04 16:00 AEST (10.4 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 70 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.626 < floor 50.0
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 39,226 tokens context.headroom_min 91,731 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 70 calls {"per_min": 7.0, "requests": 54}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 38.6 tok/s window 1: 48.84 tok/s window 2: 45.07 tok/s window 3: 41.96 tok/s window 4: 39.7 tok/s min 39.7 · max 48.84 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 209, "min_at_generation_s": 267.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 45.4 tok/s {"generation_seconds": 328.55, "generated_tokens": 14907.4}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 49.5, "requests": 11, "tool_calls": 16, "completion_tokens": 2006, "occupied_max": 5874}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 550.5, "requests": 44, "tool_calls": 54, "completion_tokens": 12968, "occupied_max": 39226}] listtasks_over_64k [] instance idscompletion_tokens 14,974 tokens {"finish_reasons": {"tool_calls": 54}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 70 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 38.6 tok/s, whole run 45.4 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e17f05543581ac460774a08531a895566f2d209d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T054940Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-stability.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012
20261004T060007Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab ✓ pass lab T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 Started 2026-10-04 16:00 AEST Finished 2026-10-04 16:13 AEST (13.3 min) Exit status 0 Notes served=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T053844Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210.json sha256 75dbaf82a0bd6bfa005b5ab1a0c00ed1eddb44fb5fe1060d7cbbd834d962f115MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 57.1 tok/s lab_prefill_tps 608 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e17f05543581ac460774a08531a895566f2d209d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T060007Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2
20261004T061329Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-load ✓ pass load T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 Started 2026-10-04 16:13 AEST Finished 2026-10-04 16:14 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T061329Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192.json sha256 6a9c08474b62518d1e1c1d23f6b6b1f241e7a0e05c5faa5f176e3700eec9fefeMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 48.9 s host_ram_drop_gb 5.84 GB vram_ready_mib 22,186 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 5af5149471babca3e64d893eeea35a3a1a43fe06 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T061329Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2
20261004T061418Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-lab ! fail lab T20b eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 Started 2026-10-04 16:14 AEST Finished 2026-10-04 16:23 AEST (9.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T061329Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192.json sha256 6a9c08474b62518d1e1c1d23f6b6b1f241e7a0e05c5faa5f176e3700eec9fefeMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 9.10 tok/s lab_prefill_tps 370 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dd520bfd96d949a57962b56ed6cd8dd55f834916 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T061418Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2
20261004T062318Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-ttft ✓ pass TTFT T20b eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 Started 2026-10-04 16:23 AEST Finished 2026-10-04 16:30 AEST (6.7 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T061329Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192.json sha256 6a9c08474b62518d1e1c1d23f6b6b1f241e7a0e05c5faa5f176e3700eec9fefeMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --batch-size 8192 --ubatch-size 8192env CUDA_DEVICE_ORDER=PCI_BUS_ID
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 384 tok/s {"prompt_tokens": [10746, 10734, 10753], "ttft_s": [27.965, 27.98, 27.965]}ttft_prefill_32k_tps 404 tok/s {"prompt_tokens": [42688, 42881, 42819], "ttft_s": [105.798, 105.946, 106.334]}ttft_prefill_tps 384 tok/s {"prompt_tokens": [10746, 10734, 10753], "ttft_s": [27.965, 27.98, 27.965]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dd520bfd96d949a57962b56ed6cd8dd55f834916 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T062318Z-llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2-ttft.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012
20261004T063003Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-load ✓ pass load T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 Started 2026-10-04 16:30 AEST Finished 2026-10-04 16:30 AEST (0.9 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T063003Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210.json sha256 e0b729222e80461545b33c6e10b8ff1d8ecaaf2fde1596824ce2a3e98b0ce76cMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 54.9 s host_ram_drop_gb 4.93 GB vram_ready_mib 65,688 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9a178e2fe77cc647e9542e1742a52c38931f07e1 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T063003Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-load.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012
20261004T063058Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-lab ! fail lab T21 eligibility: none failed stage: gates
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 Started 2026-10-04 16:30 AEST Finished 2026-10-04 16:48 AEST (17.9 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T063003Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210.json sha256 e0b729222e80461545b33c6e10b8ff1d8ecaaf2fde1596824ce2a3e98b0ce76cMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 11.2 tok/s lab_prefill_tps 258 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9a178e2fe77cc647e9542e1742a52c38931f07e1 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T063058Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-lab.json
Runs / GLM-5.3-Flash / llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012
20261004T064850Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-ttft ✓ pass TTFT T21 eligibility: none
Config llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 Started 2026-10-04 16:48 AEST Finished 2026-10-04 16:58 AEST (9.3 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T063003Z-lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-g --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210.json sha256 e0b729222e80461545b33c6e10b8ff1d8ecaaf2fde1596824ce2a3e98b0ce76cMTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --batch-size 4096 --ubatch-size 4096 --fit-target 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 298 tok/s {"prompt_tokens": [10829, 10677, 10700], "ttft_s": [37.303, 35.77, 35.957]}ttft_prefill_32k_tps 288 tok/s {"prompt_tokens": [42788, 42675, 42983], "ttft_s": [148.453, 148.434, 149.751]}ttft_prefill_tps 298 tok/s {"prompt_tokens": [10829, 10677, 10700], "ttft_s": [37.303, 35.77, 35.957]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9a178e2fe77cc647e9542e1742a52c38931f07e1 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T064850Z-llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012-ttft.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
20261004T065809Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-load ✓ pass load T22c eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 Started 2026-10-04 16:58 AEST Finished 2026-10-04 16:58 AEST (0.7 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 44.3 s host_ram_drop_gb 65.5 GB vram_ready_mib 68,156 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T065809Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-load.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
20261004T065853Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab ! fail lab T22c eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 Started 2026-10-04 16:58 AEST Finished 2026-10-04 17:10 AEST (12.0 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 14.0 tok/s lab_prefill_tps 185 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T065853Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
20261004T071053Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-ttft ✓ pass TTFT T22c eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 Started 2026-10-04 17:10 AEST Finished 2026-10-04 17:21 AEST (10.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 283 tok/s {"prompt_tokens": [10728, 10711, 10734], "ttft_s": [37.814, 37.788, 37.935]}ttft_prefill_32k_tps 253 tok/s {"prompt_tokens": [42827, 42804, 42935], "ttft_s": [167.744, 169.791, 169.931]}ttft_prefill_tps 283 tok/s {"prompt_tokens": [10728, 10711, 10734], "ttft_s": [37.814, 37.788, 37.935]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T071053Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-ttft.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
20261004T072115Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-stability ! fail stability T22c eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 Started 2026-10-04 17:21 AEST Finished 2026-10-04 17:31 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 68 tool-call parse failure(s); decode window min 13.255 < floor 15.0
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 1,874 tokens context.headroom_min 129,132 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 0 calls {"per_min": 0.0, "requests": 68}parse_failures 68 responses {"wire": {"unparsed_markup": 68}, "harness_format_errors": {"no_tool_call": 68}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 68 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 13.3 tok/s window 1: 13.31 tok/s window 2: 13.29 tok/s window 3: 13.29 tok/s window 4: 13.35 tok/s window 5: 13.34 tok/s min 13.29 · max 13.35 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 299, "min_at_generation_s": 112.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 13.3 tok/s {"generation_seconds": 418.663, "generated_tokens": 5576.2}tasks_attempted 23 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 30.9, "requests": 3, "tool_calls": 0, "completion_tokens": 246, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 27.9, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 37.7, "requests": 3, "tool_calls": 0, "completion_tokens": 348, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 31.6, "requests": 3, "tool_calls": 0, "completion_tokens": 263, "occupied_max": 1874}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 31.2, "requests": 3, "tool_calls": 0, "completion_tokens": 260, "occupied_max": 1770}, {"attempt": 6, "round": 1, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.1, "requests": 3, "tool_calls": 0, "completion_tokens": 181, "occupied_max": 1716}, {"attempt": 7, "round": 1, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 27.6, "requests": 3, "tool_calls": 0, "completion_tokens": 214, "occupied_max": 1641}, {"attempt": 8, "round": 1, "instance_id": "sympy__sympy-20590", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 39.7, "requests": 3, "tool_calls": 0, "completion_tokens": 375, "occupied_max": 1578}, {"attempt": 9, "round": 2, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 23.2, "requests": 3, "tool_calls": 0, "completion_tokens": 240, "occupied_max": 1559}, {"attempt": 10, "round": 2, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 21.1, "requests": 3, "tool_calls": 0, "completion_tokens": 212, "occupied_max": 1641}, {"attempt": 11, "round": 2, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 26.7, "requests": 3, "tool_calls": 0, "completion_tokens": 286, "occupied_max": 1609}, {"attempt": 12, "round": 2, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.0, "requests": 3, "tool_calls": 0, "completion_tokens": 262, "occupied_max": 1874}, {"attempt": 13, "round": 2, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 24.6, "requests": 3, "tool_calls": 0, "completion_tokens": 257, "occupied_max": 1770}, {"attempt": 14, "round": 2, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 19.0, "requests": 3, "tool_calls": 0, "completion_tokens": 184, "occupied_max": 1716}, {"attempt": 15, "round": 2, "instance_id": "pytest-dev__pytest-7373", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 20.8, "requests": 3, "tool_calls": 0, "completion_tokens": 208, "occupied_max": 1641}, {"attempt": 16, "round": 2, "instance_id": "sympy__sympy-20590", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 29.2, "requests": 3, "tool_calls": 0, "completion_tokens": 319, "occupied_max": 1578}, {"attempt": 17, "round": 3, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 23.2, "requests": 3, "tool_calls": 0, "completion_tokens": 240, "occupied_max": 1559}, {"attempt": 18, "round": 3, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 21.1, "requests": 3, "tool_calls": 0, "completion_tokens": 212, "occupied_max": 1641}, {"attempt": 19, "round": 3, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 26.8, "requests": 3, "tool_calls": 0, "completion_tokens": 286, "occupied_max": 1609}, {"attempt": 20, "round": 3, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 25.0, "requests": 3, "tool_calls": 0, "completion_tokens": 262, "occupied_max": 1874}, {"attempt": 21, "round": 3, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 24.6, "requests": 3, "tool_calls": 0, "completion_tokens": 257, "occupied_max": 1770}, {"attempt": 22, "round": 3, "instance_id": "astropy__astropy-12907", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 19.0, "requests": 3, "tool_calls": 0, "completion_tokens": 184, "occupied_max": 1716}, {"attempt": 23, "round": 3, "instance_id": "pytest-dev__pytest-7373", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 19.0, "requests": 3, "tool_calls": 0, "completion_tokens": 133, "occupied_max": 1522}] listtasks_over_64k [] instance idscompletion_tokens 5,646 tokens {"finish_reasons": {"stop": 68}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 0 tool calls, 68 parse failures, 0 repetition hits, tasks solved 0/23, decode min window 13.3 tok/s, whole run 13.3 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T072115Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-stability.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012
20261004T073119Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab ! fail lab T22c eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 Started 2026-10-04 17:31 AEST Finished 2026-10-04 17:42 AEST (11.2 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! lab ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T065809Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096.json sha256 fe1c037653b578219259df5ac2f7fce9deed7442d4759cd749f0001ffe3df341MTP / speculative off argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --fit-margin 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 13.4 tok/s lab_prefill_tps 189 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 981b077ba6f48c9b94c9bc0a253f95413b94b11e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T073119Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012
20261004T074233Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012-load ✕ crash load T22c eligibility: none failed stage: load
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012 Started 2026-10-04 17:42 AEST Finished 2026-10-04 17:43 AEST (0.9 min) Exit status 1 Notes c^KV (q8_0): 68.00 MiB, kv^T: not used
llama_init_from_model: KV self indexer size = 64.00 MiB (f16)
llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2754.25 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 2888039424
llama_init_from_model: failed to allocate compute buffers
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
common_speculative_init: failed to create MTP context
srv init: failed to initialize recurrent speculative context
~ggml_backend_cuda_context: have 13 graphs
~ggml_backend_cuda_context: have 14 graphs
~ggml_backend_cuda_context: have 11 graphs
Launch recorded by tools/run.py (artifacts/runs/20261004T074233Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 4096 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096.json sha256 d21078e38ec9e96da581e9003a2aaae041591909413dbf70738357df0829341eMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds unavailable not produced (crash/load) host_ram_drop_gb unavailable not produced (crash/load) vram_ready_mib unavailable not produced (crash/load)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit f3cbbe1b98ba109e9c4abaac8d34bc7b6e2775ae dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T074233Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012-load.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012
20261004T080048Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-load ✓ pass load T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 Started 2026-10-04 18:00 AEST Finished 2026-10-04 18:01 AEST (0.7 min) Exit status None Notes –
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T080048Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048.json sha256 e882737bdeeadb764f93582d803f25626630b8b0a09fb20716d71d1c2c7be516MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 39.9 s host_ram_drop_gb 3.74 GB vram_ready_mib 65,396 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 65a204856ba6e28ca395fa6aab47a88e41c62a1a dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T080048Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-load.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012
20261004T080128Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-lab ✓ pass lab T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 Started 2026-10-04 18:01 AEST Finished 2026-10-04 18:06 AEST (5.5 min) Exit status 0 Notes served=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T080048Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048.json sha256 e882737bdeeadb764f93582d803f25626630b8b0a09fb20716d71d1c2c7be516MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 44.3 tok/s lab_prefill_tps 717 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 65a204856ba6e28ca395fa6aab47a88e41c62a1a dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T080128Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-lab.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012
20261004T080657Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-ttft ✓ pass TTFT T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012 Started 2026-10-04 18:06 AEST Finished 2026-10-04 18:10 AEST (3.6 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T080048Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048.json sha256 e882737bdeeadb764f93582d803f25626630b8b0a09fb20716d71d1c2c7be516MTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 2048 --ubatch-size 2048env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 1,067 tok/s {"prompt_tokens": [13695, 13734, 13847], "ttft_s": [12.963, 12.872, 12.965]}ttft_prefill_32k_tps 931 tok/s {"prompt_tokens": [54728, 54685, 54604], "ttft_s": [58.434, 58.71, 58.906]}ttft_prefill_tps 1,067 tok/s {"prompt_tokens": [13695, 13734, 13847], "ttft_s": [12.963, 12.872, 12.965]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 65a204856ba6e28ca395fa6aab47a88e41c62a1a dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T080657Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub2048-gpu012-ttft.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012
20261004T081034Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-load ✓ pass load T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 Started 2026-10-04 18:10 AEST Finished 2026-10-04 18:11 AEST (0.7 min) Exit status None Notes –
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T081034Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096.json sha256 bb5c0e11a6559307e48f7f8f285d3439f9c7872614fa95ee033b2d48ca370dbdMTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 39.9 s host_ram_drop_gb 4.34 GB vram_ready_mib 65,508 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bfcf5657ea4ed56f452dc532e3311cf77750c3d9 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T081034Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-load.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012
20261004T081114Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-lab ✓ pass lab T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 Started 2026-10-04 18:11 AEST Finished 2026-10-04 18:19 AEST (8.1 min) Exit status 0 Notes served=/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T081034Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096.json sha256 bb5c0e11a6559307e48f7f8f285d3439f9c7872614fa95ee033b2d48ca370dbdMTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 35.0 tok/s lab_prefill_tps 713 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bfcf5657ea4ed56f452dc532e3311cf77750c3d9 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T081114Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-lab.json
Runs / Qwen3.8-Flash-Next / llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012
20261004T081919Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-ttft ✓ pass TTFT T11 eligibility: none
Config llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012 Started 2026-10-04 18:19 AEST Finished 2026-10-04 18:23 AEST (3.8 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261004T081034Z-lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210/docker-run.txt) Copy
docker run -d --name lab-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-Qwen3.8-Flash-Next-IQ4_XS:/models:ro --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5 --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096 Image ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312@sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Digest sha256:6cdf9529493b9581c421fec5628dcd4b8dd7c09507fc7d27ad739cc9205d66f5Engine llama.cpp Engine source – Launch file launches/llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096.json sha256 bb5c0e11a6559307e48f7f8f285d3439f9c7872614fa95ee033b2d48ca370dbdMTP / speculative off argv /app/llama-server --model /models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf --ctx-size 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --fit on --parallel 1 --threads 15 --jinja --no-mmproj --metrics --host 0.0.0.0 --port 8080 --split-mode layer --fit-target 2048 --batch-size 4096 --ubatch-size 4096env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/Qwen3.8-Flash-Next-GGUF @928589fdb66c6ff07f22ac561e3fbce76553548f · manifest models/bartowski-Qwen3.8-Flash-Next-IQ4_XS.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 980 tok/s {"prompt_tokens": [13680, 13707, 13810], "ttft_s": [14.13, 13.965, 14.096]}ttft_prefill_32k_tps 881 tok/s {"prompt_tokens": [54557, 54627, 54737], "ttft_s": [61.779, 62.069, 62.105]}ttft_prefill_tps 980 tok/s {"prompt_tokens": [13680, 13707, 13810], "ttft_s": [14.13, 13.965, 14.096]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bfcf5657ea4ed56f452dc532e3311cf77750c3d9 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T081919Z-llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-ub4096-gpu012-ttft.json
Runs / none
20261004T091142Z-hw-bw-gpu1-current-hw_bw ✓ pass hw bw T05a eligibility: none
Config hw-bw-gpu1-current Started 2026-10-04 19:11 AEST Finished 2026-10-04 19:11 AEST (0.2 min) Exit status 0 Notes 1 GiB pinned transfers; full table in detail
Launch no container: static measurement
Metrics context.configured – context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run h2d_gbps 13.5 GB/s {"1MB_pin": 13.14, "1MB_page": 13.01, "4MB_pin": 13.39, "4MB_page": 13.12, "16MB_pin": 13.45, "16MB_page": 13.18, "64MB_pin": 13.47, "64MB_page": 13.18, "256MB_pin": 13.47, "256MB_page": 13.07, "1024MB_pin": 13.47, "1024MB_page": 13.12}d2h_gbps 13.2 GB/s {"1MB_pin": 12.95, "1MB_page": 6.3, "4MB_pin": 13.14, "4MB_page": 10.05, "16MB_pin": 13.2, "16MB_page": 11.16, "64MB_pin": 13.21, "64MB_page": 11.79, "256MB_pin": 13.21, "256MB_page": 12.04, "1024MB_pin": 13.22, "1024MB_page": 12.05}host_memcpy_gbps 16.6 GB/s
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 00709a719d4c6057fbeb8f52eadd20c2bbe2f46d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/none/20261004T091142Z-hw-bw-gpu1-current-hw_bw.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
20261004T091154Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-load ✓ pass load T22c eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 Started 2026-10-04 19:11 AEST Finished 2026-10-04 19:12 AEST (0.8 min) Exit status None Notes –
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06cMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 48.6 s host_ram_drop_gb 83.3 GB vram_ready_mib 64,680 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit c36c7e58741670e5e869f6857478b1e198932e9a dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T091154Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-load.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
20261004T091243Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-lab ! fail lab T22c eligibility: none failed stage: gates
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 Started 2026-10-04 19:12 AEST Finished 2026-10-04 19:26 AEST (13.5 min) Exit status 0 Notes served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06cMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": false, "context": true, "speed": false}lab_decode_c1_tps 10.2 tok/s lab_prefill_tps 166 tok/s "context-gate prompt tokens / total request seconds"mtp_acceptance_rate 0.5137 fraction {"draft_tokens": 3366, "accepted_tokens": 1729, "source": "server log delta"}mtp_accepted_tps 10.2 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 151350a4d65da543200768e08741f6eb5409c3ed dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T091243Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-lab.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
20261004T092615Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-ttft ✓ pass TTFT T22c eligibility: none
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 Started 2026-10-04 19:26 AEST Finished 2026-10-04 19:38 AEST (11.7 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06cMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 248 tok/s {"prompt_tokens": [10798, 10674, 10665], "ttft_s": [43.785, 43.033, 42.927]}ttft_prefill_32k_tps 224 tok/s {"prompt_tokens": [42970, 42867, 42822], "ttft_s": [191.654, 191.974, 191.365]}ttft_prefill_tps 248 tok/s {"prompt_tokens": [10798, 10674, 10665], "ttft_s": [43.785, 43.033, 42.927]}
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 42d955c4fa51a99398f09218c4a4c133bcf18938 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T092615Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-ttft.json
Runs / GLM-5.3-Flash / ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012
20261004T093800Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-stability ✕ crash stability T22c eligibility: none failed stage: measure
Config ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 Started 2026-10-04 19:38 AEST Finished 2026-10-04 19:48 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation)
Other runs of this config: load lab ! TTFT stability ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T091154Z-lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012/docker-run.txt) Copy
docker run -d --name lab-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 --security-opt no-new-privileges --gpus '"device=GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d,GPU-c67ac872-3371-a88c-aa92-d969daf6405e,GPU-76d3c6af-7f18-69d2-3145-899103de1722"' -p 127.0.0.1:18080:8080 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e CUDA_VISIBLE_DEVICES=2,1,0 -v ~/models/local-ai-exp/bartowski-GLM-5.3-Flash-IQ2_M:/models:ro --entrypoint /app/llama-server local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1 --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168 Image local-ai-bench/ik-llama-cuda@sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Digest sha256:d30fa2796c72509c9158a6202fed46c8ce159f3344f3735120be65a73a8137b1Engine ik_llama.cpp Engine source https://github.com/ikawrakow/ik_llama.cpp/tree/5bf8f0fe4db98e9560de0f495232f8d3a733c678 Launch file launches/ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168.json sha256 cf6116ffb2e3698333ab48e1435f99a6f037b39308f84e62e4fd24888017f06cMTP / speculative {"type": "mtp", "draft": "embedded NextN/MTP layer of the target GGUF (Q4_0 in bartowski files)", "spec_type": "mtp:n_max=3,p_min=0.0", "n_max": 3, "p_min": 0.0, "heads": 1, "ref": "ik_llama.cpp PR #2548 (n_max=3 best in author's n_max 3/4 comparison)"}argv /app/llama-server --model /models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf --ctx-size 131072 --flash-attn on --mla-use 3 --dsa --cache-type-k q8_0 --n-gpu-layers 999 --fit --split-mode layer --batch-size 2048 --ubatch-size 2048 --parallel 1 --threads 15 --jinja --metrics --host 0.0.0.0 --port 8080 --spec-type mtp:n_max=3,p_min=0.0 --fit-margin 7168env CUDA_DEVICE_ORDER=PCI_BUS_IDCUDA_VISIBLE_DEVICES=2,1,0
bartowski/GLM-5.3-Flash-BF16-GGUF @66e5e9f0e337470e8801b99bae6c45779b042cc2 · manifest models/bartowski-GLM-5.3-Flash-IQ2_M.json
Metrics context.configured 131,072 tokens context.occupied_max 1,641 tokens context.headroom_min 129,336 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 0 calls {"per_min": 0.0, "requests": 9}parse_failures 9 responses {"wire": {"unparsed_markup": 9}, "harness_format_errors": {"no_tool_call": 9}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 9 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps unavailable only 41.172 s of generation; need > 120 s for one window decode_whole_run_tps 15.6 tok/s {"generation_seconds": 41.172, "generated_tokens": 642.7}tasks_attempted 5 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom yes {"upstream_connection_failures": 54, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 26.8, "requests": 3, "tool_calls": 0, "completion_tokens": 208, "occupied_max": 1559}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 28.3, "requests": 3, "tool_calls": 0, "completion_tokens": 227, "occupied_max": 1641}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "seconds": 27.1, "requests": 3, "tool_calls": 0, "completion_tokens": 217, "occupied_max": 1609}, {"attempt": 4, "round": 1, "instance_id": "matplotlib__matplotlib-23299", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 343.6}, {"attempt": 5, "round": 1, "instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 174.2}] listtasks_over_64k [] instance idscompletion_tokens 652 tokens {"finish_reasons": {"stop": 9}}mtp_accepted_tps unavailable not produced (crash/measure) mtp_acceptance_rate unavailable not produced (crash/measure)
Agentic summary: 600 s, 0 tool calls, 9 parse failures, 0 repetition hits, tasks solved 0/5, decode min window – tok/s, whole run 15.6 tok/s.
Cache, layout and host Cache warm Layout current · 3 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 ● NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 ● NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 42d955c4fa51a99398f09218c4a4c133bcf18938 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T093800Z-ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012-stability.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T101015Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ✓ pass load T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:10 AEST Finished 2026-10-04 20:14 AEST (3.9 min) Exit status None Notes –
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 236 s host_ram_drop_gb 63.5 GB vram_ready_mib 20,818 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bdd5d466ac35145adc76b9f18538d7ef1373acce dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T101015Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T101411Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ✓ pass lab T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:14 AEST Finished 2026-10-04 20:19 AEST (5.4 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.6 tok/s lab_prefill_tps 2,433 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bdd5d466ac35145adc76b9f18538d7ef1373acce dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T101411Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T101937Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ✓ pass kit sweep T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:19 AEST Finished 2026-10-04 20:34 AEST (15.2 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,289 tok/s {"min": 2269.6, "max": 2294.4, "n": 3}prefill_16k_tps 2,240 tok/s {"min": 2233.0, "max": 2247.3, "n": 3}prefill_32k_tps 2,678 tok/s {"min": 2672.6, "max": 2681.2, "n": 3}prefill_64k_tps 2,653 tok/s {"min": 2645.1, "max": 2655.6, "n": 3}decode_c1_tps 48.2 tok/s {"min": 47.55, "max": 48.85, "power_w": null}decode_c2_tps 53.1 tok/s {"min": 51.57, "max": 54.71, "power_w": null}decode_c3_tps 55.9 tok/s {"min": 55.26, "max": 56.61, "power_w": null}decode_c4_tps 53.3 tok/s {"min": 51.51, "max": 55.06, "power_w": null}decode_c1_32k_tps 49.9 tok/s {"min": 49.58, "max": 50.2, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T101937Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T103452Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ✓ pass TTFT T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:34 AEST Finished 2026-10-04 20:36 AEST (1.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,017 tok/s {"prompt_tokens": [13801, 13745, 13788], "ttft_s": [6.842, 6.83, 6.822]}ttft_prefill_32k_tps 2,687 tok/s {"prompt_tokens": [54693, 54557, 54874], "ttft_s": [20.353, 20.312, 20.371]}ttft_prefill_tps 2,017 tok/s {"prompt_tokens": [13801, 13745, 13788], "ttft_s": [6.842, 6.83, 6.822]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T103452Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T103614Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ✓ pass quality panel T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:36 AEST Finished 2026-10-04 20:36 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.98867 fraction mean_kl 0.00096647 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T103614Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T103641Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ! fail stability T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:36 AEST Finished 2026-10-04 20:47 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 67 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.939 < floor 50.0
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 36,327 tokens context.headroom_min 168,259 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 67 calls {"per_min": 6.7, "requests": 46}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 37.9 tok/s window 1: 39.97 tok/s window 2: 43.13 tok/s window 3: 42.36 tok/s window 4: 46.42 tok/s window 5: 43.39 tok/s window 6: 42.38 tok/s min 39.97 · max 46.42 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 313, "min_at_generation_s": 87.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 44.1 tok/s {"generation_seconds": 432.859, "generated_tokens": 19093.9}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 216.5, "requests": 17, "tool_calls": 29, "completion_tokens": 6657, "occupied_max": 13921}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 383.5, "requests": 30, "tool_calls": 38, "completion_tokens": 12541, "occupied_max": 36327}] listtasks_over_64k [] instance idscompletion_tokens 19,198 tokens {"finish_reasons": {"tool_calls": 46}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 67 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 37.9 tok/s, whole run 44.1 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T103641Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2
20261004T104713Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72 ✓ pass lab T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 Started 2026-10-04 20:47 AEST Finished 2026-10-04 20:52 AEST (5.3 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T101015Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Digest sha256:f77c72f659567d28ffa2804806f69624e227530b4870a3d609f166014af1b451Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6.json sha256 1671d4e04532370bdb34ebf4958454f7b186984f5668d6c90538d416b0bca440MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 54.5 tok/s lab_prefill_tps 2,596 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9c490cfb769bb4bce8cd979c9e96a84128be7bc8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T104713Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2
20261004T105250Z-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2-load ✕ crash load T15 eligibility: none failed stage: load
Config sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2 Started 2026-10-04 20:52 AEST Finished 2026-10-04 20:55 AEST (3.0 min) Exit status 0 Notes _glue/offload_moe_method.py", line 86, in get_runtime
store = HostExpertStore(model_dir, layers, num_experts, hidden, inter, bits, prefix=prefix)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/trellis-serve/cuda/src/sglang_exl3/kernels/offload_store.py", line 62, in __init__
raise NotImplementedError("HostExpertStore: fast relayout implemented for K=3 experts only")
NotImplementedError: HostExpertStore: fast relayout implemented for K=3 experts only
[2026-10-04 10:55:46] Received sigquit from a child process. It usually means the child failed.
[2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=True, pid=1
[2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Launch recorded by tools/run.py (artifacts/runs/20261004T105250Z-lab-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4:/models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb.json sha256 0d170921f6e46f05cad846a1404d158b6e85e4fffe18eeec9a2d7d70ec79a969MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @65c895314393431c09050b2e04e250836b3a6eb4 · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-2.05bpw_h4_ng4.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds unavailable not produced (crash/load) host_ram_drop_gb unavailable not produced (crash/load) vram_ready_mib unavailable not produced (crash/load)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit df0d10f31ae46a27f371fda8e1f15e40dfa878f0 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T105250Z-sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2
20261004T105602Z-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2-load ✕ crash load T15 eligibility: none failed stage: load
Config sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2 Started 2026-10-04 20:56 AEST Finished 2026-10-04 20:56 AEST (0.9 min) Exit status 0 Notes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 116, in __init__
t = parse_table(model_path)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 94, in parse_table
raise ValueError(f"{path}: shard_N.trellis tensors missing or not contiguous")
ValueError: /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6/ngram_embedding.safetensors: shard_N.trellis tensors missing or not contiguous
[2026-10-04 10:56:54] Received sigquit from a child process. It usually means the child failed.
[2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=True, pid=1
[2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Launch recorded by tools/run.py (artifacts/runs/20261004T105602Z-lab-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6:/models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Digest sha256:ce47df25cee7ff235d8bfdf3e79e3828aa11bee3624db3ce121ebc79a8375dd3Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/1e72b250cb529c6d81bf11b88740d9a0b22f62de Launch file launches/sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb.json sha256 ce41d62d49f95e441684dbb7afcbfd2cd54c0cdf49476dda70cda1eca5bc7928MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @55a732e0c4c3d4614bc42b68493bb930d9b02c0a · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds unavailable not produced (crash/load) host_ram_drop_gb unavailable not produced (crash/load) vram_ready_mib unavailable not produced (crash/load)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit bd62e89c96f97db97945fd89be01017a19fea7c3 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T105602Z-sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2-load.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T105700Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load ✓ pass load T23 eligibility: none
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 20:57 AEST Finished 2026-10-04 21:02 AEST (5.0 min) Exit status None Notes –
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 300 s host_ram_drop_gb 120 GB vram_ready_mib 23,322 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T105700Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T110201Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab ! fail lab T23 eligibility: none failed stage: gates
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 21:02 AEST Finished 2026-10-04 21:12 AEST (11.0 min) Exit status 0 Notes served=glm-5.3-flash
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 7.40 tok/s lab_prefill_tps 739 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T110201Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T111300Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft ! fail TTFT T23 eligibility: none failed stage: invalid measurement
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 21:13 AEST Finished 2026-10-04 21:17 AEST (4.2 min) Exit status 0 Notes INVALIDATED: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2. headline length 8192; prompts carry random nonces
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps unavailable invalidated: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2 ttft_prefill_32k_tps unavailable invalidated: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2 ttft_prefill_tps unavailable invalidated: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T111300Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T111712Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-stability ! fail stability T23 eligibility: none
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 21:17 AEST Finished 2026-10-04 21:27 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 21 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 3.356 < floor 15.0
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max 4,589 tokens context.headroom_min 126,405 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 21 calls {"per_min": 2.1, "requests": 14}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 3.36 tok/s window 1: 6.08 tok/s window 2: 5.96 tok/s min 5.96 · max 6.08 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 99, "min_at_generation_s": 101.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 6.56 tok/s {"generation_seconds": 218.025, "generated_tokens": 1431.1}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 385.2, "requests": 11, "tool_calls": 16, "completion_tokens": 1605, "occupied_max": 4589}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 214.8, "requests": 4, "tool_calls": 5, "completion_tokens": 179, "occupied_max": 2222}] listtasks_over_64k [] instance idscompletion_tokens 1,784 tokens {"finish_reasons": {"tool_calls": 14}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 21 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 3.36 tok/s, whole run 6.56 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T111712Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-stability.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T112744Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab ! fail lab T23 eligibility: none failed stage: gates
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 21:27 AEST Finished 2026-10-04 22:05 AEST (37.9 min) Exit status 0 Notes served=glm-5.3-flash
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T105700Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 7.50 tok/s lab_prefill_tps 720 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d36a5ce7e63d3abd393f2511b2a55930e67bf549 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T112744Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T120556Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:05 AEST Finished 2026-10-04 22:10 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 260 s host_ram_drop_gb 65.9 GB vram_ready_mib 20,822 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T120556Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T121018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:10 AEST Finished 2026-10-04 22:15 AEST (4.7 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.8 tok/s lab_prefill_tps 2,419 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T121018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T121501Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:15 AEST Finished 2026-10-04 22:30 AEST (15.7 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,281 tok/s {"min": 2261.8, "max": 2289.8, "n": 3}prefill_16k_tps 2,237 tok/s {"min": 2234.0, "max": 2244.7, "n": 3}prefill_32k_tps 2,678 tok/s {"min": 2670.7, "max": 2679.3, "n": 3}prefill_64k_tps 2,649 tok/s {"min": 2643.2, "max": 2654.6, "n": 3}decode_c1_tps 47.4 tok/s {"min": 46.48, "max": 48.27, "power_w": null}decode_c2_tps 53.1 tok/s {"min": 51.34, "max": 54.86, "power_w": null}decode_c3_tps 60.6 tok/s {"min": 57.47, "max": 63.7, "power_w": null}decode_c4_tps 53.7 tok/s {"min": 52.62, "max": 54.77, "power_w": null}decode_c1_32k_tps 48.5 tok/s {"min": 48.52, "max": 48.57, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T121501Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T123041Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:30 AEST Finished 2026-10-04 22:32 AEST (1.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,009 tok/s {"prompt_tokens": [13741, 13755, 13738], "ttft_s": [6.864, 6.84, 6.838]}ttft_prefill_32k_tps 2,681 tok/s {"prompt_tokens": [54658, 54711, 54684], "ttft_s": [20.392, 20.395, 20.396]}ttft_prefill_tps 2,009 tok/s {"prompt_tokens": [13741, 13755, 13738], "ttft_s": [6.864, 6.84, 6.838]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T123041Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T123204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality panel T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:32 AEST Finished 2026-10-04 22:32 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.98967 fraction mean_kl 0.00097327 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T123204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T123230Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✕ crash stability T13 eligibility: none failed stage: measure
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:32 AEST Finished 2026-10-04 22:42 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation)
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T120556Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no successful request context.headroom_min unavailable configured context unknown (set CONFIGURED_CTX) duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 0 calls {"per_min": 0.0, "requests": 0}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": null, "saved": "artifacts: proxy/repetition/"}decode_window_min_tps unavailable only 0.0 s of generation; need > 120 s for one window decode_whole_run_tps unavailable no generation recorded tasks_attempted 3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom yes {"upstream_connection_failures": 77, "upstream_http_errors": 1, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 267.3}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 261.8}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 71.0}] listtasks_over_64k [] instance idscompletion_tokens 0 tokens {"finish_reasons": {}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 0 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/3, decode min window – tok/s, whole run – tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dcbaa681afe8032d6e59244d27f56918b1e7bd20 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T123230Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2
20261004T124252Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 Started 2026-10-04 22:42 AEST Finished 2026-10-04 22:47 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load lab ✕ TTFT ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T124252Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6.json sha256 90d5526c99bf2c3b87f25e51276eed60130a38e3a34bc2991695048a8bb33eb9MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 257 s host_ram_drop_gb 65.3 GB vram_ready_mib 21,706 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit db208909a5ae27d314ff09d53d7cf803d67fda9c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T124252Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2
20261004T124710Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✕ crash lab T14a eligibility: none failed stage: harness
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 Started 2026-10-04 22:47 AEST Finished 2026-10-04 22:48 AEST (1.7 min) Exit status 1 Notes –
Other runs of this config: load lab ✕ TTFT ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T124252Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6.json sha256 90d5526c99bf2c3b87f25e51276eed60130a38e3a34bc2991695048a8bb33eb9MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed unavailable not produced (crash/harness) lab_decode_c1_tps unavailable not produced (crash/harness) lab_prefill_tps unavailable not produced (crash/harness) mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit db208909a5ae27d314ff09d53d7cf803d67fda9c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T124710Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2
20261004T124852Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✕ crash TTFT T14a eligibility: none failed stage: measure
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 Started 2026-10-04 22:48 AEST Finished 2026-10-04 22:49 AEST (0.3 min) Exit status 1 Notes –
Other runs of this config: load lab ✕ TTFT ✕
Launch recorded by tools/run.py (artifacts/runs/20261004T124252Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6.json sha256 90d5526c99bf2c3b87f25e51276eed60130a38e3a34bc2991695048a8bb33eb9MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=6.0SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_tps unavailable not produced (crash/measure)
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit db208909a5ae27d314ff09d53d7cf803d67fda9c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T124852Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T124925Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load ✓ pass load T23 eligibility: none
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 22:49 AEST Finished 2026-10-04 22:54 AEST (4.6 min) Exit status None Notes –
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T124925Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 276 s host_ram_drop_gb 121 GB vram_ready_mib 23,322 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e2424ec92249689799954b79c057d915d7f402f1 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T124925Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-load.json
Runs / GLM-5.3-Flash / exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2
20261004T125401Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft ✓ pass TTFT T23 eligibility: none
Config exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 Started 2026-10-04 22:54 AEST Finished 2026-10-04 22:58 AEST (4.2 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load lab ! lab ! TTFT ! TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T124925Z-lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2/docker-run.txt) Copy
docker run -d --name lab-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e GLM53_MODE=exact -e GLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -e GLM53_MODEL_DOWNLOAD=0 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models/turboderp-GLM-5.3-Flash-exl3-3.05bpw:ro --entrypoint /opt/glm53/docker/entrypoint.sh ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551d -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flash Image ghcr.io/0xsero/glm53-flash-offload@sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dDigest sha256:bb633b0bcb85573ad40b6c408af5e1036ae062c593e42479f4e4d2ab521e551dEngine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/fe97bcf846347d035859a0610cebbb707fa36caa Launch file launches/exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x.json sha256 b679beb9863b299d0b6e44b1269b954e5292fb8039caa75a0e4c0d34e6cb17faMTP / speculative off argv /opt/glm53/docker/entrypoint.sh -m /models/turboderp-GLM-5.3-Flash-exl3-3.05bpw -cs 131072 --max-batch-size 8 -chunk_size 8192 -ambs 4 --host 0.0.0.0 --port 30000 --served-name glm-5.3-flashenv GLM53_MODE=exactGLM53_MODEL_DIR=/models/turboderp-GLM-5.3-Flash-exl3-3.05bpwGLM53_MODEL_DOWNLOAD=0HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 483 tok/s {"prompt_tokens": [10753, 10793, 10717], "ttft_s": [19.173, 22.332, 25.068]}ttft_prefill_32k_tps 688 tok/s {"prompt_tokens": [42865, 42692, 42811], "ttft_s": [62.271, 62.501, 60.752]}ttft_prefill_tps 483 tok/s {"prompt_tokens": [10753, 10793, 10717], "ttft_s": [19.173, 22.332, 25.068]}
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e2424ec92249689799954b79c057d915d7f402f1 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261004T125401Z-exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T125827Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 22:58 AEST Finished 2026-10-04 23:02 AEST (4.4 min) Exit status None Notes –
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T125827Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 264 s host_ram_drop_gb 65.7 GB vram_ready_mib 20,844 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9229192eace16fc1782f479846e0194aea593ba9 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T125827Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T130251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 23:02 AEST Finished 2026-10-04 23:13 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.676 < floor 50.0
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T125827Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 25,309 tokens context.headroom_min 178,942 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 71 calls {"per_min": 7.1, "requests": 47}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 39.7 tok/s window 1: 44.31 tok/s window 2: 43.52 tok/s window 3: 43.73 tok/s window 4: 44.87 tok/s window 5: 41.92 tok/s min 41.92 · max 44.87 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 254, "min_at_generation_s": 267.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.3 tok/s {"generation_seconds": 373.189, "generated_tokens": 16176.9}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 245.6, "requests": 21, "tool_calls": 31, "completion_tokens": 5704, "occupied_max": 13986}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 354.4, "requests": 27, "tool_calls": 40, "completion_tokens": 10614, "occupied_max": 25309}] listtasks_over_64k [] instance idscompletion_tokens 16,318 tokens {"finish_reasons": {"tool_calls": 47}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 71 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.7 tok/s, whole run 43.3 tok/s.
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9229192eace16fc1782f479846e0194aea593ba9 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T130251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T131323Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-04 23:13 AEST Finished 2026-10-04 23:22 AEST (8.9 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T125827Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 57.4 tok/s lab_prefill_tps 2,537 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9229192eace16fc1782f479846e0194aea593ba9 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T131323Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
20261004T132232Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 Started 2026-10-04 23:22 AEST Finished 2026-10-04 23:26 AEST (4.2 min) Exit status None Notes –
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 254 s host_ram_drop_gb 64.4 GB vram_ready_mib 20,744 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 97294db30283c264f2c58088bb36bfb1619ba3de dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T132232Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
20261004T132647Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 Started 2026-10-04 23:26 AEST Finished 2026-10-04 23:31 AEST (5.0 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.6 tok/s lab_prefill_tps 2,430 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 97294db30283c264f2c58088bb36bfb1619ba3de dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T132647Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
20261004T133146Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 Started 2026-10-04 23:31 AEST Finished 2026-10-04 23:33 AEST (1.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,014 tok/s {"prompt_tokens": [13689, 13759, 13721], "ttft_s": [6.848, 6.829, 6.812]}ttft_prefill_32k_tps 2,692 tok/s {"prompt_tokens": [54692, 54821, 54730], "ttft_s": [20.316, 20.367, 20.356]}ttft_prefill_tps 2,014 tok/s {"prompt_tokens": [13689, 13759, 13721], "ttft_s": [6.848, 6.829, 6.812]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 97294db30283c264f2c58088bb36bfb1619ba3de dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T133146Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
20261004T133309Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 Started 2026-10-04 23:33 AEST Finished 2026-10-04 23:43 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 57 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.08 < floor 50.0
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 29,027 tokens context.headroom_min 174,880 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 57 calls {"per_min": 5.7, "requests": 42}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 40.1 tok/s window 1: 42.01 tok/s window 2: 46.09 tok/s window 3: 40.09 tok/s window 4: 43.71 tok/s min 40.09 · max 46.09 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 193, "min_at_generation_s": 181.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.1 tok/s {"generation_seconds": 312.161, "generated_tokens": 13447.2}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 124.1, "requests": 11, "tool_calls": 18, "completion_tokens": 3257, "occupied_max": 8494}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 475.9, "requests": 32, "tool_calls": 39, "completion_tokens": 10303, "occupied_max": 29027}] listtasks_over_64k [] instance idscompletion_tokens 13,560 tokens {"finish_reasons": {"tool_calls": 42}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 57 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 40.1 tok/s, whole run 43.1 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 97294db30283c264f2c58088bb36bfb1619ba3de dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T133309Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2
20261004T134340Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 Started 2026-10-04 23:43 AEST Finished 2026-10-04 23:48 AEST (4.6 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T132232Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1.json sha256 e76072403dd6ef64022ca8b6ccb17b7e7d25f20aa7c17b8287c7534158eee184MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 1 --cuda-graph-max-bs-decode 1 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 54.6 tok/s lab_prefill_tps 2,580 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 97294db30283c264f2c58088bb36bfb1619ba3de dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T134340Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
20261004T134832Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14b eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Started 2026-10-04 23:48 AEST Finished 2026-10-04 23:52 AEST (3.6 min) Exit status None Notes –
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=pinnedSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 215 s host_ram_drop_gb 88.2 GB vram_ready_mib 20,740 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T134832Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
20261004T135208Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14b eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Started 2026-10-04 23:52 AEST Finished 2026-10-04 23:55 AEST (3.7 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=pinnedSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 53.7 tok/s lab_prefill_tps 2,440 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T135208Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
20261004T135547Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T14b eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Started 2026-10-04 23:55 AEST Finished 2026-10-04 23:57 AEST (1.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=pinnedSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,022 tok/s {"prompt_tokens": [13765, 13739, 13758], "ttft_s": [6.807, 6.819, 6.799]}ttft_prefill_32k_tps 2,700 tok/s {"prompt_tokens": [54781, 54752, 54783], "ttft_s": [20.259, 20.275, 20.301]}ttft_prefill_tps 2,022 tok/s {"prompt_tokens": [13765, 13739, 13758], "ttft_s": [6.807, 6.819, 6.799]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T135547Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
20261004T135709Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14b eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Started 2026-10-04 23:57 AEST Finished 2026-10-05 00:07 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 72 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.915 < floor 50.0
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=pinnedSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 39,281 tokens context.headroom_min 165,395 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 72 calls {"per_min": 7.2, "requests": 50}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 40.9 tok/s window 1: 46.31 tok/s window 2: 42.86 tok/s window 3: 45.63 tok/s window 4: 44.46 tok/s window 5: 47.12 tok/s min 42.86 · max 47.12 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 289, "min_at_generation_s": 108.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 44.6 tok/s {"generation_seconds": 408.606, "generated_tokens": 18207.3}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 137.5, "requests": 16, "tool_calls": 22, "completion_tokens": 3550, "occupied_max": 10227}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 462.8, "requests": 34, "tool_calls": 50, "completion_tokens": 14781, "occupied_max": 39281}] listtasks_over_64k [] instance idscompletion_tokens 18,331 tokens {"finish_reasons": {"tool_calls": 50}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 72 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 40.9 tok/s, whole run 44.6 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T135709Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2
20261004T140742Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14b eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 Started 2026-10-05 00:07 AEST Finished 2026-10-05 00:11 AEST (3.7 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab TTFT stability !
Launch recorded by tools/run.py (artifacts/runs/20261004T134832Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=pinned -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned.json sha256 06f6cc13b0c25dfb2d50a8b515c6240bf1418dfc8b07facf231d3e0847358cb1MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=pinnedSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 51.3 tok/s lab_prefill_tps 2,596 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display off hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 off 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1b7b92a2ae73d921b9b6ab2ff7cb2059734faf95 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T140742Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T141422Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 00:14 AEST Finished 2026-10-05 00:18 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T141422Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 257 s host_ram_drop_gb 64.5 GB vram_ready_mib 20,818 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 742e81596c31f25802bdf11347fc6d9ab1bb8265 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T141422Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261004T141839Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 00:18 AEST Finished 2026-10-05 00:25 AEST (7.0 min) Exit status 0 Notes official lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb.200k.20261004T142537.json; recipe written only on pass
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T141422Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.8 tok/s lab_prefill_tps 2,419 tok/s "official lab.py try"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 742e81596c31f25802bdf11347fc6d9ab1bb8265 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T141839Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T225552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 08:55 AEST Finished 2026-10-05 09:00 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 260 s host_ram_drop_gb 64.7 GB vram_ready_mib 19,890 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T225552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T230013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 09:00 AEST Finished 2026-10-05 09:04 AEST (4.2 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 48.1 tok/s lab_prefill_tps 2,408 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T230013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T230426Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 09:04 AEST Finished 2026-10-05 09:20 AEST (16.2 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,257 tok/s {"min": 2238.2, "max": 2266.1, "n": 3}prefill_16k_tps 2,207 tok/s {"min": 2201.0, "max": 2213.3, "n": 3}prefill_32k_tps 2,644 tok/s {"min": 2643.9, "max": 2652.5, "n": 3}prefill_64k_tps 2,619 tok/s {"min": 2611.0, "max": 2620.8, "n": 3}decode_c1_tps 43.7 tok/s {"min": 42.8, "max": 44.57, "power_w": null}decode_c2_tps 49.0 tok/s {"min": 48.07, "max": 49.85, "power_w": null}decode_c3_tps 50.8 tok/s {"min": 48.26, "max": 53.38, "power_w": null}decode_c4_tps 49.5 tok/s {"min": 48.16, "max": 50.79, "power_w": null}decode_c1_32k_tps 45.5 tok/s {"min": 45.12, "max": 45.97, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T230426Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T232036Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 09:20 AEST Finished 2026-10-05 09:21 AEST (1.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 1,969 tok/s {"prompt_tokens": [13696, 13698, 13802], "ttft_s": [6.977, 6.955, 6.951]}ttft_prefill_32k_tps 2,650 tok/s {"prompt_tokens": [54741, 54745, 54708], "ttft_s": [20.622, 20.657, 20.663]}ttft_prefill_tps 1,969 tok/s {"prompt_tokens": [13696, 13698, 13802], "ttft_s": [6.977, 6.955, 6.951]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T232036Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T232159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality panel T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 09:21 AEST Finished 2026-10-05 09:22 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.98834 fraction mean_kl 0.00099656 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T232159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T232226Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 09:22 AEST Finished 2026-10-05 09:33 AEST (10.6 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.429 < floor 50.0
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 34,417 tokens context.headroom_min 170,165 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 71 calls {"per_min": 7.1, "requests": 45}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 37.4 tok/s window 1: 39.3 tok/s window 2: 40.68 tok/s window 3: 45.52 tok/s window 4: 38.23 tok/s window 5: 41.18 tok/s min 38.23 · max 45.52 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 260, "min_at_generation_s": 263.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 41.1 tok/s {"generation_seconds": 379.256, "generated_tokens": 15573.9}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 255.0, "requests": 22, "tool_calls": 36, "completion_tokens": 6757, "occupied_max": 13743}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 345.0, "requests": 23, "tool_calls": 35, "completion_tokens": 8934, "occupied_max": 34417}] listtasks_over_64k [] instance idscompletion_tokens 15,691 tokens {"finish_reasons": {"tool_calls": 45}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 71 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 37.4 tok/s, whole run 41.1 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T232226Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2
20261004T233302Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 Started 2026-10-05 09:33 AEST Finished 2026-10-05 09:37 AEST (4.3 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep TTFT stability ! quality panel
Launch recorded by tools/run.py (artifacts/runs/20261004T225552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926.json sha256 0dd86341b226c6d31aacb7a1409e736274bb7bfb5e0658581e7fdcf091510bafMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=9.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 49.7 tok/s lab_prefill_tps 2,564 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 33d67751778fe52a0111a696912cb037e739d585 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T233302Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261004T233738Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 09:37 AEST Finished 2026-10-05 09:41 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 260 s host_ram_drop_gb 64.3 GB vram_ready_mib 20,818 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T233738Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261004T234159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 09:41 AEST Finished 2026-10-05 09:49 AEST (8.0 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 54.9 tok/s lab_prefill_tps 2,131 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T234159Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261004T234959Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 09:49 AEST Finished 2026-10-05 10:05 AEST (15.1 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,270 tok/s {"min": 2269.2, "max": 2290.0, "n": 3}prefill_16k_tps 2,404 tok/s {"min": 2390.5, "max": 2411.1, "n": 3}prefill_32k_tps 2,436 tok/s {"min": 2433.8, "max": 2438.6, "n": 3}prefill_64k_tps 2,424 tok/s {"min": 2423.1, "max": 2429.3, "n": 3}decode_c1_tps 48.2 tok/s {"min": 47.15, "max": 49.33, "power_w": null}decode_c2_tps 53.7 tok/s {"min": 52.77, "max": 54.63, "power_w": null}decode_c3_tps 56.2 tok/s {"min": 55.37, "max": 57.07, "power_w": null}decode_c4_tps 53.1 tok/s {"min": 51.6, "max": 54.52, "power_w": null}decode_c1_32k_tps 48.8 tok/s {"min": 47.71, "max": 49.89, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261004T234959Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T000508Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 10:05 AEST Finished 2026-10-05 10:06 AEST (1.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,294 tok/s {"prompt_tokens": [13720, 13775, 13799], "ttft_s": [6.023, 6.006, 6.007]}ttft_prefill_32k_tps 2,369 tok/s {"prompt_tokens": [54807, 54652, 54690], "ttft_s": [23.113, 23.087, 23.088]}ttft_prefill_tps 2,294 tok/s {"prompt_tokens": [13720, 13775, 13799], "ttft_s": [6.023, 6.006, 6.007]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T000508Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T000636Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality panel T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 10:06 AEST Finished 2026-10-05 10:07 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.98934 fraction mean_kl 0.00091506 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T000636Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T000703Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 10:07 AEST Finished 2026-10-05 10:17 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 64 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.539 < floor 50.0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 25,179 tokens context.headroom_min 178,831 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 64 calls {"per_min": 6.4, "requests": 47}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 39.5 tok/s window 1: 43.49 tok/s window 2: 46.17 tok/s window 3: 44.39 tok/s window 4: 41.1 tok/s min 41.1 · max 46.17 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 238, "min_at_generation_s": 72.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.9 tok/s {"generation_seconds": 357.957, "generated_tokens": 15726.5}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 202.1, "requests": 22, "tool_calls": 28, "completion_tokens": 5212, "occupied_max": 13779}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 397.9, "requests": 26, "tool_calls": 36, "completion_tokens": 10647, "occupied_max": 25179}] listtasks_over_64k [] instance idscompletion_tokens 15,859 tokens {"finish_reasons": {"tool_calls": 47}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 64 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.5 tok/s, whole run 43.9 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T000703Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T001735Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 10:17 AEST Finished 2026-10-05 10:21 AEST (3.7 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261004T233738Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 49.5 tok/s lab_prefill_tps 2,237 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 745dba2bc47b01ae057b745e4a5d72db2dc1bd4c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T001735Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261005T002133Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 10:21 AEST Finished 2026-10-05 10:25 AEST (4.4 min) Exit status None Notes –
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 263 s host_ram_drop_gb 65.3 GB vram_ready_mib 20,822 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit c04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T002133Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261005T002557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 10:25 AEST Finished 2026-10-05 10:32 AEST (6.1 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 49.9 tok/s lab_prefill_tps 2,419 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit c04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T002557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261005T003204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 10:32 AEST Finished 2026-10-05 10:47 AEST (15.2 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,268 tok/s {"min": 2267.6, "max": 2289.9, "n": 3}prefill_16k_tps 2,241 tok/s {"min": 2237.2, "max": 2243.2, "n": 3}prefill_32k_tps 2,669 tok/s {"min": 2668.5, "max": 2679.3, "n": 3}prefill_64k_tps 2,648 tok/s {"min": 2642.8, "max": 2649.4, "n": 3}decode_c1_tps 47.8 tok/s {"min": 46.55, "max": 49.04, "power_w": null}decode_c2_tps 54.5 tok/s {"min": 52.67, "max": 56.29, "power_w": null}decode_c3_tps 55.0 tok/s {"min": 53.94, "max": 56.08, "power_w": null}decode_c4_tps 54.3 tok/s {"min": 52.99, "max": 55.54, "power_w": null}decode_c1_32k_tps 49.7 tok/s {"min": 49.39, "max": 50.06, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit c04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T003204Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261005T004714Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 10:47 AEST Finished 2026-10-05 10:48 AEST (1.4 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,021 tok/s {"prompt_tokens": [13841, 13799, 13715], "ttft_s": [6.844, 6.829, 6.841]}ttft_prefill_32k_tps 2,685 tok/s {"prompt_tokens": [54709, 54687, 54693], "ttft_s": [20.355, 20.37, 20.381]}ttft_prefill_tps 2,021 tok/s {"prompt_tokens": [13841, 13799, 13715], "ttft_s": [6.844, 6.829, 6.841]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit c04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T004714Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261005T004836Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality panel T13 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 10:48 AEST Finished 2026-10-05 10:49 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.991 fraction mean_kl 0.00095858 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit c04100985eb79c66cc3d3bfe61c60a0dc8ecf0ec dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T004836Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2
20261005T004903Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✕ crash stability T13 eligibility: none failed stage: measure
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 Started 2026-10-05 10:49 AEST Finished 2026-10-05 10:59 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 6 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation)
Other runs of this config: load load load load lab lab lab lab kit sweep kit sweep TTFT TTFT stability ✕ stability ! stability ✕ quality panel quality panel
Launch recorded by tools/run.py (artifacts/runs/20261005T002133Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6.json sha256 cbcfae235adc9e4c7988a047150225a710b8e9ed2b2370701c5d2a9820c28a01MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 12288 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 4,566 tokens context.headroom_min 200,016 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 6 calls {"per_min": 0.6, "requests": 4}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps unavailable only 21.565 s of generation; need > 120 s for one window decode_whole_run_tps 43.0 tok/s {"generation_seconds": 21.565, "generated_tokens": 927.9}tasks_attempted 3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom yes {"upstream_connection_failures": 74, "upstream_http_errors": 1, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 303.1, "requests": 4, "tool_calls": 6, "completion_tokens": 942, "occupied_max": 4566}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Error:BadGatewayError: litellm.BadGatewayError: BadGatewayError: OpenAIException - Cannot connect to host 127.0.0.1:18080 ssl:default [Connect call failed ('127.0.0.1', 18080)]", "submitted": false, "resolved": false, "seconds": 261.1}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 35.8}] listtasks_over_64k [] instance idscompletion_tokens 942 tokens {"finish_reasons": {"tool_calls": 4}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 6 tool calls, 0 parse failures, 0 repetition hits, tasks solved 0/3, decode min window – tok/s, whole run 43.0 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 99ed822bca74427cce1bd0d05509c9693385510b dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T004903Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T005928Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 10:59 AEST Finished 2026-10-05 11:03 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 257 s host_ram_drop_gb 64.5 GB vram_ready_mib 20,818 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T005928Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T010346Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:03 AEST Finished 2026-10-05 11:10 AEST (7.1 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 53.5 tok/s lab_prefill_tps 2,134 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T010346Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T011051Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:10 AEST Finished 2026-10-05 11:25 AEST (15.1 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,287 tok/s {"min": 2264.6, "max": 2295.6, "n": 3}prefill_16k_tps 2,406 tok/s {"min": 2405.0, "max": 2412.2, "n": 3}prefill_32k_tps 2,440 tok/s {"min": 2439.7, "max": 2445.4, "n": 3}prefill_64k_tps 2,432 tok/s {"min": 2426.3, "max": 2433.4, "n": 3}decode_c1_tps 47.0 tok/s {"min": 46.18, "max": 47.84, "power_w": null}decode_c2_tps 53.5 tok/s {"min": 51.42, "max": 55.58, "power_w": null}decode_c3_tps 56.4 tok/s {"min": 55.16, "max": 57.55, "power_w": null}decode_c4_tps 53.5 tok/s {"min": 51.17, "max": 55.94, "power_w": null}decode_c1_32k_tps 48.3 tok/s {"min": 47.04, "max": 49.6, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T011051Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T012559Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:25 AEST Finished 2026-10-05 11:27 AEST (1.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,283 tok/s {"prompt_tokens": [13674, 13731, 13722], "ttft_s": [6.012, 6.004, 6.01]}ttft_prefill_32k_tps 2,369 tok/s {"prompt_tokens": [54628, 54734, 54799], "ttft_s": [23.071, 23.104, 23.118]}ttft_prefill_tps 2,283 tok/s {"prompt_tokens": [13674, 13731, 13722], "ttft_s": [6.012, 6.004, 6.01]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T012559Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T012727Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality panel T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:27 AEST Finished 2026-10-05 11:27 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.98934 fraction mean_kl 0.00090651 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T012727Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T012753Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:27 AEST Finished 2026-10-05 11:38 AEST (10.6 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 62 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.784 < floor 50.0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 28,209 tokens context.headroom_min 176,303 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 62 calls {"per_min": 6.2, "requests": 49}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 38.8 tok/s window 1: 41.79 tok/s window 2: 44.66 tok/s window 3: 44.2 tok/s window 4: 39.85 tok/s min 39.85 · max 44.66 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 225, "min_at_generation_s": 247.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.2 tok/s {"generation_seconds": 344.575, "generated_tokens": 14882.0}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 137.4, "requests": 15, "tool_calls": 21, "completion_tokens": 3375, "occupied_max": 10259}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 462.6, "requests": 35, "tool_calls": 42, "completion_tokens": 11629, "occupied_max": 28209}] listtasks_over_64k [] instance idscompletion_tokens 15,004 tokens {"finish_reasons": {"tool_calls": 49}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 62 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 38.8 tok/s, whole run 43.2 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T012753Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T013828Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:38 AEST Finished 2026-10-05 11:45 AEST (7.1 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T005928Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 56.4 tok/s lab_prefill_tps 2,228 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit d3125c68b10d6f03dba10710bce202f1a355c61f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T013828Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T014552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:45 AEST Finished 2026-10-05 11:50 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 260 s host_ram_drop_gb 65.2 GB vram_ready_mib 20,822 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T014552Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T015013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:50 AEST Finished 2026-10-05 11:56 AEST (6.7 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.0 tok/s lab_prefill_tps 2,131 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T015013Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T015658Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:56 AEST Finished 2026-10-05 11:58 AEST (1.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,289 tok/s {"prompt_tokens": [13839, 13729, 13726], "ttft_s": [6.033, 6.005, 5.997]}ttft_prefill_32k_tps 2,379 tok/s {"prompt_tokens": [54743, 54741, 54693], "ttft_s": [23.003, 23.011, 23.009]}ttft_prefill_tps 2,289 tok/s {"prompt_tokens": [13839, 13729, 13726], "ttft_s": [6.033, 6.005, 5.997]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T015658Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T015826Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality panel T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:58 AEST Finished 2026-10-05 11:58 AEST (0.4 min) Exit status 0 Notes panel=qwen3.8-flash-next-exl3-ref-panel.json rc=0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 0.99167 fraction mean_kl 0.0009967 nats in_band yes "top1>=0.987, KL<=0.0013"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T015826Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T015853Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 11:58 AEST Finished 2026-10-05 12:13 AEST (14.9 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,292 tok/s {"min": 2283.1, "max": 2303.8, "n": 3}prefill_16k_tps 2,400 tok/s {"min": 2395.7, "max": 2402.2, "n": 3}prefill_32k_tps 2,434 tok/s {"min": 2429.8, "max": 2436.1, "n": 3}prefill_64k_tps 2,421 tok/s {"min": 2419.3, "max": 2424.7, "n": 3}decode_c1_tps 48.1 tok/s {"min": 46.11, "max": 50.12, "power_w": null}decode_c2_tps 53.7 tok/s {"min": 51.67, "max": 55.76, "power_w": null}decode_c3_tps 55.7 tok/s {"min": 53.92, "max": 57.53, "power_w": null}decode_c4_tps 53.8 tok/s {"min": 52.2, "max": 55.37, "power_w": null}decode_c1_32k_tps 49.6 tok/s {"min": 48.96, "max": 50.18, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T015853Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T021344Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 12:13 AEST Finished 2026-10-05 12:24 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 61 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.37 < floor 50.0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 30,539 tokens context.headroom_min 174,142 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 61 calls {"per_min": 6.1, "requests": 39}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 40.4 tok/s window 1: 41.91 tok/s window 2: 46.32 tok/s window 3: 43.26 tok/s window 4: 44.94 tok/s window 5: 43.94 tok/s min 41.91 · max 46.32 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 252, "min_at_generation_s": 311.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 44.0 tok/s {"generation_seconds": 371.065, "generated_tokens": 16312.6}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 171.3, "requests": 16, "tool_calls": 25, "completion_tokens": 4565, "occupied_max": 10307}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 428.7, "requests": 24, "tool_calls": 36, "completion_tokens": 11840, "occupied_max": 30539}] listtasks_over_64k [] instance idscompletion_tokens 16,405 tokens {"finish_reasons": {"tool_calls": 39}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 61 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 40.4 tok/s, whole run 44.0 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T021344Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T022416Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T14a eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 12:24 AEST Finished 2026-10-05 12:29 AEST (4.9 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T014552Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 45.0 tok/s lab_prefill_tps 2,240 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89b76f527155eabc290c2443970efffc5809929c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T022416Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T080528Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-load ✓ pass load T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:05 AEST Finished 2026-10-05 18:07 AEST (2.1 min) Exit status None Notes –
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 128 s host_ram_drop_gb 7.43 GB vram_ready_mib 22,038 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit f6465c1b9998f35f8a63466311b53a40b12cb385 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T080528Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-load.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T080737Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab ! fail lab T40 eligibility: none failed stage: gates
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:07 AEST Finished 2026-10-05 18:08 AEST (0.9 min) Exit status 1 Notes –
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run mtp_acceptance_rate 0.73 fraction {"mean_accept_len": 3.192, "batches": 4, "source": "SGLang decode log"}mtp_accepted_tps unavailable no decode speed in this step gates_passed unavailable not produced (fail/gates) lab_decode_c1_tps unavailable not produced (fail/gates) lab_prefill_tps unavailable not produced (fail/gates)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit f6465c1b9998f35f8a63466311b53a40b12cb385 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T080737Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T080832Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-kit_sweep ✓ pass kit sweep T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:08 AEST Finished 2026-10-05 18:21 AEST (12.7 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 1,076 tok/s {"min": 1074.8, "max": 1077.8, "n": 3}prefill_16k_tps 1,038 tok/s {"min": 1037.3, "max": 1041.5, "n": 3}prefill_32k_tps 970 tok/s {"min": 969.7, "max": 970.8, "n": 3}prefill_64k_tps 854 tok/s {"min": 854.0, "max": 854.6, "n": 3}decode_c1_tps 121 tok/s {"min": 119.17, "max": 122.69, "power_w": null}decode_c2_tps 122 tok/s {"min": 120.82, "max": 122.99, "power_w": null}decode_c3_tps 122 tok/s {"min": 120.77, "max": 123.86, "power_w": null}decode_c4_tps 122 tok/s {"min": 121.62, "max": 122.8, "power_w": null}decode_c1_32k_tps 112 tok/s {"min": 106.7, "max": 116.43, "power_w": null}mtp_acceptance_rate 0.5626 fraction {"mean_accept_len": 2.688, "batches": 362, "source": "SGLang decode log"}mtp_accepted_tps 121 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 26d933974ee1425a2fcec22a974c63c651bfa53c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T080832Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-kit_sweep.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T082114Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-ttft ✓ pass TTFT T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:21 AEST Finished 2026-10-05 18:25 AEST (3.8 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 1,035 tok/s {"prompt_tokens": [13746, 13684, 13729], "ttft_s": [13.273, 13.223, 13.268]}ttft_prefill_32k_tps 883 tok/s {"prompt_tokens": [54766, 54807, 54718], "ttft_s": [62.035, 62.071, 61.932]}ttft_prefill_tps 1,035 tok/s {"prompt_tokens": [13746, 13684, 13729], "ttft_s": [13.273, 13.223, 13.268]}mtp_acceptance_rate 0.66 fraction {"mean_accept_len": 2.98, "batches": 1, "source": "SGLang decode log"}mtp_accepted_tps unavailable no decode speed in this step
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 26d933974ee1425a2fcec22a974c63c651bfa53c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T082114Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-ttft.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T082500Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-needle ! fail needle T40 eligibility: none failed stage: measure
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:25 AEST Finished 2026-10-05 18:25 AEST (0.1 min) Exit status 0 Notes kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max 0 tokens context.headroom_min unavailable no occupancy measured in this run needle_pass_rate 0 fraction [{"target_tokens": 131072, "depth": 0.2, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}, {"target_tokens": 131072, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}, {"target_tokens": 131072, "depth": 0.8, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}, {"target_tokens": 196608, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]needle_max_passed_tokens 0 tokens
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 26d933974ee1425a2fcec22a974c63c651bfa53c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T082500Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-needle.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T082503Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-stability ! fail stability T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:25 AEST Finished 2026-10-05 18:35 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 137 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s)
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max 52,045 tokens context.headroom_min 152,447 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 137 calls {"per_min": 13.7, "requests": 108}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 1 hits {"ngram64x3": 1, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 111 tok/s window 1: 120.94 tok/s window 2: 116.4 tok/s window 3: 113.65 tok/s window 4: 131.93 tok/s window 5: 116.65 tok/s window 6: 113.23 tok/s min 113.23 · max 131.93 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 322, "min_at_generation_s": 342.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 120 tok/s {"generation_seconds": 441.794, "generated_tokens": 52884.0}tasks_attempted 3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 2 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 53.7, "requests": 15, "tool_calls": 20, "completion_tokens": 4610, "occupied_max": 12441}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 304.1, "requests": 44, "tool_calls": 56, "completion_tokens": 25509, "occupied_max": 52045}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 242.5, "requests": 49, "tool_calls": 61, "completion_tokens": 23352, "occupied_max": 35168}] listtasks_over_64k [] instance idscompletion_tokens 53,471 tokens {"finish_reasons": {"tool_calls": 108}}mtp_acceptance_rate 0.6711 fraction {"mean_accept_len": 3.013, "batches": 446, "source": "SGLang decode log"}mtp_accepted_tps 120 tok/s "output tok/s including accepted draft tokens"
Agentic summary: 600 s, 137 tool calls, 0 parse failures, 1 repetition hits, tasks solved 2/3, decode min window 111 tok/s, whole run 120 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 7601cd0b9171268d7ab9b01200decc2680ca4987 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T082503Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-stability.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T083536Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab ! fail lab T40 eligibility: none failed stage: gates
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:35 AEST Finished 2026-10-05 18:36 AEST (0.9 min) Exit status 1 Notes –
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run mtp_acceptance_rate 0.71 fraction {"mean_accept_len": 3.123, "batches": 4, "source": "SGLang decode log"}mtp_accepted_tps unavailable no decode speed in this step gates_passed unavailable not produced (fail/gates) lab_decode_c1_tps unavailable not produced (fail/gates) lab_prefill_tps unavailable not produced (fail/gates)
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 7601cd0b9171268d7ab9b01200decc2680ca4987 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T083536Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-lab.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2
20261005T083628Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-quality_ev ✓ pass quality eval T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 Started 2026-10-05 18:36 AEST Finished 2026-10-05 21:06 AEST (150.0 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load lab ! lab ! kit sweep TTFT stability ! quality eval needle !
Launch recorded by tools/run.py (artifacts/runs/20261005T080528Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy.json sha256 c4b825a12a5c77c6e174f31bce8236f5a51644c6ec2b576ab3760ca54d6ee925MTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max 77,769 tokens context.headroom_min 126,604 tokens tasks_attempted 30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 18 tasks {"eval_errors": 0}solve_rate 0.6 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21395, "budget_exhausted": false, "seconds": 302.2, "requests": 38, "tool_calls": 52, "occupied_max": 35932}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19992, "budget_exhausted": false, "seconds": 212.1, "requests": 36, "tool_calls": 40, "occupied_max": 35374}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 54056, "budget_exhausted": false, "seconds": 533.9, "requests": 53, "tool_calls": 69, "occupied_max": 66916}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33030, "budget_exhausted": false, "seconds": 347.7, "requests": 34, "tool_calls": 44, "occupied_max": 56721}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9909, "budget_exhausted": false, "seconds": 102.3, "requests": 24, "tool_calls": 29, "occupied_max": 19157}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27576, "budget_exhausted": false, "seconds": 279.2, "requests": 56, "tool_calls": 69, "occupied_max": 40391}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 4996, "budget_exhausted": false, "seconds": 114.0, "requests": 22, "tool_calls": 32, "occupied_max": 19356}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 16378, "budget_exhausted": false, "seconds": 176.0, "requests": 36, "tool_calls": 36, "occupied_max": 27395}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10825, "budget_exhausted": false, "seconds": 108.7, "requests": 23, "tool_calls": 35, "occupied_max": 24167}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29907, "budget_exhausted": false, "seconds": 331.6, "requests": 49, "tool_calls": 72, "occupied_max": 48615}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 28779, "budget_exhausted": false, "seconds": 252.5, "requests": 27, "tool_calls": 34, "occupied_max": 33418}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 14338, "budget_exhausted": false, "seconds": 168.3, "requests": 40, "tool_calls": 57, "occupied_max": 35704}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 11456, "budget_exhausted": false, "seconds": 129.7, "requests": 30, "tool_calls": 41, "occupied_max": 21732}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 50913, "budget_exhausted": false, "seconds": 657.2, "requests": 79, "tool_calls": 86, "occupied_max": 77769}, {"instance_id": "sympy__sympy-12171", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 506.5, "requests": 33, "tool_calls": 40, "occupied_max": 35466}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 48319, "budget_exhausted": false, "seconds": 543.6, "requests": 80, "tool_calls": 89, "occupied_max": 75240}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7993, "budget_exhausted": false, "seconds": 122.3, "requests": 29, "tool_calls": 32, "occupied_max": 19839}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11280, "budget_exhausted": false, "seconds": 116.3, "requests": 23, "tool_calls": 32, "occupied_max": 22150}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14524, "budget_exhausted": false, "seconds": 232.2, "requests": 47, "tool_calls": 61, "occupied_max": 29338}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 10764, "budget_exhausted": false, "seconds": 121.4, "requests": 34, "tool_calls": 42, "occupied_max": 23921}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 19349, "budget_exhausted": false, "seconds": 210.6, "requests": 45, "tool_calls": 55, "occupied_max": 35457}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 36683, "budget_exhausted": false, "seconds": 377.3, "requests": 57, "tool_calls": 75, "occupied_max": 49046}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25193, "budget_exhausted": false, "seconds": 237.0, "requests": 45, "tool_calls": 52, "occupied_max": 33408}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34857, "budget_exhausted": false, "seconds": 382.7, "requests": 35, "tool_calls": 42, "occupied_max": 41142}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 2815, "budget_exhausted": false, "seconds": 32.9, "requests": 14, "tool_calls": 18, "occupied_max": 7743}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 481.7, "requests": 32, "tool_calls": 38, "occupied_max": 36017}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6628, "budget_exhausted": false, "seconds": 75.5, "requests": 24, "tool_calls": 32, "occupied_max": 14570}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 54124, "budget_exhausted": false, "seconds": 518.1, "requests": 39, "tool_calls": 49, "occupied_max": 57827}, {"instance_id": "django__django-16408", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 575.9, "requests": 49, "tool_calls": 63, "occupied_max": 69131}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24950, "budget_exhausted": false, "seconds": 250.1, "requests": 30, "tool_calls": 37, "occupied_max": 28452}] listtool_calls 1,453 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 5 responses {"wire": {"invalid_tool_call": 4, "unparsed_markup": 1}, "harness_format_errors": {"truncated_no_tool_call": 20, "no_tool_call": 5, "invalid_arguments": 2}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 20 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 5 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 72 hits {"ngram64x3": 72, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}decode_whole_run_tps 125 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records mtp_acceptance_rate 0.7343 fraction {"mean_accept_len": 3.203, "batches": 6353, "source": "SGLang decode log"}mtp_accepted_tps 125 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2bcf6651904c0d286b5235b08ff4e99397b24089 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T083628Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2-quality_ev.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T110634Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:06 AEST Finished 2026-10-05 21:10 AEST (4.4 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 263 s host_ram_drop_gb 64.7 GB vram_ready_mib 20,844 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T110634Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T111058Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:10 AEST Finished 2026-10-05 21:16 AEST (5.3 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 48.8 tok/s lab_prefill_tps 2,131 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T111058Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T111618Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:16 AEST Finished 2026-10-05 21:31 AEST (15.1 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,286 tok/s {"min": 2270.2, "max": 2294.5, "n": 3}prefill_16k_tps 2,408 tok/s {"min": 2397.4, "max": 2413.9, "n": 3}prefill_32k_tps 2,444 tok/s {"min": 2440.5, "max": 2444.1, "n": 3}prefill_64k_tps 2,431 tok/s {"min": 2427.0, "max": 2433.7, "n": 3}decode_c1_tps 47.5 tok/s {"min": 45.23, "max": 49.7, "power_w": null}decode_c2_tps 54.3 tok/s {"min": 52.8, "max": 55.79, "power_w": null}decode_c3_tps 56.2 tok/s {"min": 55.2, "max": 57.26, "power_w": null}decode_c4_tps 54.3 tok/s {"min": 53.1, "max": 55.51, "power_w": null}decode_c1_32k_tps 48.5 tok/s {"min": 47.81, "max": 49.28, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T111618Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T113123Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass TTFT T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:31 AEST Finished 2026-10-05 21:32 AEST (1.5 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,284 tok/s {"prompt_tokens": [13748, 13735, 13713], "ttft_s": [6.019, 6.002, 6.005]}ttft_prefill_32k_tps 2,372 tok/s {"prompt_tokens": [54749, 54796, 54671], "ttft_s": [23.074, 23.105, 23.093]}ttft_prefill_tps 2,284 tok/s {"prompt_tokens": [13748, 13735, 13713], "ttft_s": [6.019, 6.002, 6.005]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T113123Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T113251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail needle T40 eligibility: none failed stage: measure
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:32 AEST Finished 2026-10-05 21:35 AEST (2.3 min) Exit status 0 Notes kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 104,552 tokens context.headroom_min unavailable no occupancy measured in this run needle_pass_rate 0.75 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 45.0}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 44.8}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 44.8}, {"target_tokens": 196608, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]needle_max_passed_tokens 104,552 tokens
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 915058cd8abcbcf3fb5f0083096df6cb6effba65 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T113251Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T113507Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:35 AEST Finished 2026-10-05 21:45 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 69 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.78 < floor 50.0
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 28,542 tokens context.headroom_min 175,656 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 69 calls {"per_min": 6.9, "requests": 56}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 39.8 tok/s window 1: 39.78 tok/s window 2: 45.05 tok/s window 3: 46.26 tok/s window 4: 45.84 tok/s min 39.78 · max 46.26 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 225, "min_at_generation_s": 60.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.6 tok/s {"generation_seconds": 344.08, "generated_tokens": 15015.9}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 198.6, "requests": 20, "tool_calls": 29, "completion_tokens": 5184, "occupied_max": 14275}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 401.4, "requests": 37, "tool_calls": 41, "completion_tokens": 10080, "occupied_max": 29194}] listtasks_over_64k [] instance idscompletion_tokens 15,169 tokens {"finish_reasons": {"tool_calls": 56}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 69 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.8 tok/s, whole run 43.6 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89508b66cd245628d95157a9fb3d38d309518e32 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T113507Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T114539Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:45 AEST Finished 2026-10-05 21:50 AEST (4.5 min) Exit status 0 Notes served=flashnext
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 54.6 tok/s lab_prefill_tps 2,234 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89508b66cd245628d95157a9fb3d38d309518e32 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T114539Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T115010Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass quality eval T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-05 21:50 AEST Finished 2026-10-06 03:13 AEST (323.0 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T110634Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 73,243 tokens context.headroom_min 130,698 tokens tasks_attempted 30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 30 tasks {"eval_errors": 0}solve_rate 1.00 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24487, "budget_exhausted": false, "seconds": 1282.3, "requests": 62, "tool_calls": 75, "occupied_max": 53966}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14321, "budget_exhausted": false, "seconds": 645.2, "requests": 31, "tool_calls": 37, "occupied_max": 28187}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11569, "budget_exhausted": false, "seconds": 520.6, "requests": 26, "tool_calls": 38, "occupied_max": 26802}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20399, "budget_exhausted": false, "seconds": 773.6, "requests": 47, "tool_calls": 56, "occupied_max": 49582}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9370, "budget_exhausted": false, "seconds": 335.5, "requests": 28, "tool_calls": 37, "occupied_max": 21738}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21298, "budget_exhausted": false, "seconds": 893.6, "requests": 50, "tool_calls": 70, "occupied_max": 43815}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9622, "budget_exhausted": false, "seconds": 373.8, "requests": 30, "tool_calls": 42, "occupied_max": 27935}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 38145, "budget_exhausted": false, "seconds": 1384.0, "requests": 67, "tool_calls": 90, "occupied_max": 73243}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9853, "budget_exhausted": false, "seconds": 323.5, "requests": 27, "tool_calls": 41, "occupied_max": 22641}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5844, "budget_exhausted": false, "seconds": 215.4, "requests": 22, "tool_calls": 30, "occupied_max": 13904}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12939, "budget_exhausted": false, "seconds": 472.4, "requests": 43, "tool_calls": 58, "occupied_max": 27930}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10158, "budget_exhausted": false, "seconds": 379.2, "requests": 28, "tool_calls": 39, "occupied_max": 30678}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9242, "budget_exhausted": false, "seconds": 1449.4, "requests": 45, "tool_calls": 54, "occupied_max": 24025}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12532, "budget_exhausted": false, "seconds": 877.7, "requests": 47, "tool_calls": 62, "occupied_max": 34916}, {"instance_id": "sympy__sympy-12171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18740, "budget_exhausted": false, "seconds": 572.2, "requests": 37, "tool_calls": 50, "occupied_max": 38369}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11528, "budget_exhausted": false, "seconds": 398.4, "requests": 39, "tool_calls": 49, "occupied_max": 31537}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15258, "budget_exhausted": false, "seconds": 439.0, "requests": 24, "tool_calls": 28, "occupied_max": 29043}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11438, "budget_exhausted": false, "seconds": 359.5, "requests": 27, "tool_calls": 33, "occupied_max": 21736}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27193, "budget_exhausted": false, "seconds": 911.4, "requests": 63, "tool_calls": 69, "occupied_max": 51234}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11426, "budget_exhausted": false, "seconds": 486.3, "requests": 27, "tool_calls": 41, "occupied_max": 25678}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10873, "budget_exhausted": false, "seconds": 349.6, "requests": 23, "tool_calls": 33, "occupied_max": 27369}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18642, "budget_exhausted": false, "seconds": 806.8, "requests": 42, "tool_calls": 59, "occupied_max": 43584}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22233, "budget_exhausted": false, "seconds": 774.2, "requests": 46, "tool_calls": 60, "occupied_max": 48294}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11156, "budget_exhausted": false, "seconds": 554.6, "requests": 33, "tool_calls": 39, "occupied_max": 27211}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9148, "budget_exhausted": false, "seconds": 321.8, "requests": 24, "tool_calls": 35, "occupied_max": 19416}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26215, "budget_exhausted": false, "seconds": 1133.1, "requests": 56, "tool_calls": 66, "occupied_max": 54318}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6092, "budget_exhausted": false, "seconds": 208.5, "requests": 23, "tool_calls": 26, "occupied_max": 15749}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15049, "budget_exhausted": false, "seconds": 458.2, "requests": 29, "tool_calls": 40, "occupied_max": 25926}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20168, "budget_exhausted": false, "seconds": 735.3, "requests": 44, "tool_calls": 48, "occupied_max": 41643}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10487, "budget_exhausted": false, "seconds": 389.8, "requests": 34, "tool_calls": 40, "occupied_max": 24793}] listtool_calls 1,445 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 1 responses {"wire": {"invalid_tool_call": 1}, "harness_format_errors": {"unknown_tool": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 16 hits {"ngram64x3": 15, "same_call_x4": 1, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}decode_whole_run_tps 42.3 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 89508b66cd245628d95157a9fb3d38d309518e32 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T115010Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T171331Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo ✓ pass load T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:13 AEST Finished 2026-10-06 03:15 AEST (2.1 min) Exit status None Notes –
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T171331Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 128 s host_ram_drop_gb 7.46 GB vram_ready_mib 22,038 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 3c69f81efea146b3536e73d563c29f649bf837c3 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T171331Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T171540Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la ! fail lab T40 eligibility: none failed stage: gates
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:15 AEST Finished 2026-10-06 03:16 AEST (0.9 min) Exit status 1 Notes –
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T171331Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run mtp_acceptance_rate 0.7075 fraction {"mean_accept_len": 3.125, "batches": 4, "source": "SGLang decode log"}mtp_accepted_tps unavailable no decode speed in this step gates_passed unavailable not produced (fail/gates) lab_decode_c1_tps unavailable not produced (fail/gates) lab_prefill_tps unavailable not produced (fail/gates)
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 3c69f81efea146b3536e73d563c29f649bf837c3 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T171540Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T171640Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo ✓ pass load T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:16 AEST Finished 2026-10-06 03:18 AEST (2.1 min) Exit status None Notes –
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T171640Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 128 s host_ram_drop_gb 7.39 GB vram_ready_mib 22,038 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 28a6dff9d14e1e69befacd5bacb1eb12a1e67e16 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T171640Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T171849Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne ! fail needle T40 eligibility: none failed stage: measure
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:18 AEST Finished 2026-10-06 03:25 AEST (7.1 min) Exit status 0 Notes kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T171640Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max 104,552 tokens context.headroom_min unavailable no occupancy measured in this run needle_pass_rate 0.75 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 142.1}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 139.7}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 139.9}, {"target_tokens": 180000, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]needle_max_passed_tokens 104,552 tokens
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 28a6dff9d14e1e69befacd5bacb1eb12a1e67e16 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T171849Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T172557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-06 03:25 AEST Finished 2026-10-06 03:30 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T172557Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 260 s host_ram_drop_gb 65.6 GB vram_ready_mib 20,844 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 8458078aea61608e4013c1096b9fda1d2949fa33 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T172557Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261005T173018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass needle T40 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-06 03:30 AEST Finished 2026-10-06 03:34 AEST (4.3 min) Exit status 0 Notes kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261005T172557Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 191,045 tokens context.headroom_min unavailable no occupancy measured in this run needle_pass_rate 1.00 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 80.7}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 44.7}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 44.7}, {"target_tokens": 180000, "prompt_tokens": 191045, "depth": 0.5, "code": "5568-7992", "answer": "5568-7992", "pass": true, "seconds": 86.1}]needle_max_passed_tokens 191,045 tokens
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 8458078aea61608e4013c1096b9fda1d2949fa33 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261005T173018Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T173533Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo ✓ pass load T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:35 AEST Finished 2026-10-06 03:37 AEST (2.1 min) Exit status None Notes –
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T173533Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 128 s host_ram_drop_gb 7.49 GB vram_ready_mib 22,038 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e435b7aadf7a7467836beef995d22b7c7a6ef9ae dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T173533Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T173741Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la ✓ pass lab T40 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:37 AEST Finished 2026-10-06 03:41 AEST (3.6 min) Exit status 0 Notes served=turboderp-Qwen3.8-27B-exl3-3.00bpw
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T173533Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 144 tok/s lab_prefill_tps 740 tok/s "context-gate prompt tokens / total request seconds"mtp_acceptance_rate 0.7986 fraction {"mean_accept_len": 3.398, "batches": 77, "source": "SGLang decode log"}mtp_accepted_tps 144 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e435b7aadf7a7467836beef995d22b7c7a6ef9ae dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T173741Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-la.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261005T174120Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne ! fail needle T40 eligibility: none failed stage: measure
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-06 03:41 AEST Finished 2026-10-06 03:48 AEST (7.0 min) Exit status 0 Notes kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261005T173533Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max 104,552 tokens context.headroom_min unavailable no occupancy measured in this run needle_pass_rate 0.75 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 139.9}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 140.0}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 140.2}, {"target_tokens": 180000, "depth": 0.5, "pass": false, "error": "<HTTPError 400: 'Bad Request'>"}]needle_max_passed_tokens 104,552 tokens
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e435b7aadf7a7467836beef995d22b7c7a6ef9ae dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261005T174120Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-ne.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T061419Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load ✓ pass load T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 16:14 AEST Finished 2026-10-08 16:14 AEST (0.4 min) Exit status None Notes –
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 24.9 s host_ram_drop_gb 39.2 GB vram_ready_mib 23,646 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit ee97643caed24b4da8ea676da3e385a7b4c9f6a6 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T061419Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T061444Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab ✓ pass lab T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 16:14 AEST Finished 2026-10-08 16:17 AEST (2.3 min) Exit status 0 Notes served=qwen3.8-flash-next-q2_0
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 100 tok/s lab_prefill_tps 2,664 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit f47b2f23a90940c82663797750c922ab301f7b72 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T061444Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T061705Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-kit_sweep ! fail kit sweep T41 eligibility: none failed stage: measure
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 16:17 AEST Finished 2026-10-08 16:17 AEST (0.0 min) Exit status 0 Notes sweep status: None
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps unavailable not reached prefill_16k_tps unavailable not reached prefill_32k_tps unavailable not reached prefill_64k_tps unavailable not reached decode_c1_tps unavailable not reached decode_c2_tps unavailable not reached decode_c3_tps unavailable not reached decode_c4_tps unavailable not reached decode_c1_32k_tps unavailable not reached mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit f47b2f23a90940c82663797750c922ab301f7b72 cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T061705Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-kit_sweep.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T061706Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-ttft ✓ pass TTFT T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 16:17 AEST Finished 2026-10-08 16:18 AEST (1.3 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,620 tok/s {"prompt_tokens": [13752, 13748, 13713], "ttft_s": [5.421, 5.248, 5.183]}ttft_prefill_32k_tps 2,769 tok/s {"prompt_tokens": [54762, 54790, 54685], "ttft_s": [19.776, 19.785, 19.796]}ttft_prefill_tps 2,620 tok/s {"prompt_tokens": [13752, 13748, 13713], "ttft_s": [5.421, 5.248, 5.183]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 8050e8e736925db26a1dcb695f1a33034f8e7d84 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T061706Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T061821Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-stability ✓ pass stability T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 16:18 AEST Finished 2026-10-08 16:29 AEST (10.6 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 116 tool calls in this run count toward the config's 100-call minimum (§5)
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 38,712 tokens context.headroom_min 92,079 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 116 calls {"per_min": 11.6, "requests": 97}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 123 tok/s window 1: 128.04 tok/s window 2: 124.61 tok/s window 3: 127.52 tok/s min 124.61 · max 128.04 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 151, "min_at_generation_s": 111.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 127 tok/s {"generation_seconds": 270.449, "generated_tokens": 34254.1}tasks_attempted 3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 2 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 52.3, "requests": 18, "tool_calls": 28, "completion_tokens": 3181, "occupied_max": 12800}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 240.9, "requests": 49, "tool_calls": 57, "completion_tokens": 18657, "occupied_max": 38712}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 307.1, "requests": 30, "tool_calls": 31, "completion_tokens": 12531, "occupied_max": 27651}] listtasks_over_64k [] instance idscompletion_tokens 34,369 tokens {"finish_reasons": {"tool_calls": 97}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 116 tool calls, 0 parse failures, 0 repetition hits, tasks solved 2/3, decode min window 123 tok/s, whole run 127 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 8050e8e736925db26a1dcb695f1a33034f8e7d84 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T061821Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-stability.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T062900Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab ✓ pass lab T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 16:29 AEST Finished 2026-10-08 16:32 AEST (3.5 min) Exit status 0 Notes served=qwen3.8-flash-next-q2_0
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T061419Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 100 tok/s lab_prefill_tps 2,657 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 8050e8e736925db26a1dcb695f1a33034f8e7d84 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T062900Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-lab.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
20261008T063235Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-load ✓ pass load T42 eligibility: none
Config glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 Started 2026-10-08 16:32 AEST Finished 2026-10-08 16:34 AEST (1.7 min) Exit status None Notes –
Other runs of this config: load lab ! kit sweep ! TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512MTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 101 s host_ram_drop_gb 51.4 GB vram_ready_mib 22,832 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit add6123e4cc212c9e92bff5991dcc4be26d9bf9a cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T063235Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-load.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
20261008T063417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-lab ! fail lab T42 eligibility: none failed stage: gates
Config glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 Started 2026-10-08 16:34 AEST Finished 2026-10-08 17:34 AEST (59.7 min) Exit status 0 Notes served=glm-5.3-flash
Other runs of this config: load lab ! kit sweep ! TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512MTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 6.10 tok/s lab_prefill_tps 314 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit add6123e4cc212c9e92bff5991dcc4be26d9bf9a cleanpins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T063417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-lab.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
20261008T073401Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-kit_sweep ! fail kit sweep T42 eligibility: none failed stage: timeout
Config glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 Started 2026-10-08 17:34 AEST Finished 2026-10-08 19:34 AEST (120.3 min) Exit status None Notes –
Other runs of this config: load lab ! kit sweep ! TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512MTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps unavailable not produced (fail/timeout) prefill_16k_tps unavailable not produced (fail/timeout) prefill_32k_tps unavailable not produced (fail/timeout) prefill_64k_tps unavailable not produced (fail/timeout) decode_c1_tps unavailable not produced (fail/timeout) decode_c2_tps unavailable not produced (fail/timeout) decode_c3_tps unavailable not produced (fail/timeout) decode_c4_tps unavailable not produced (fail/timeout) mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit fd5de8462f016c61391e0aacfbaef661ffe68784 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T073401Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-kit_sweep.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
20261008T093417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-quality_panel ✓ pass quality panel T42 eligibility: none
Config glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 Started 2026-10-08 19:34 AEST Finished 2026-10-08 20:25 AEST (51.3 min) Exit status 0 Notes panel=glm-5.3-flash-exl3-ref-panel.json rc=0
Other runs of this config: load lab ! kit sweep ! TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512MTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 1.00 fraction mean_kl 0 nats in_band yes "top1>=0.99, KL<=0.001"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 4188a3d929072e112d5fc4c196b1f349c8a4f0f8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T093417Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-quality_panel.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2
20261008T102534Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-ttft ✓ pass TTFT T42 eligibility: none
Config glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 Started 2026-10-08 20:25 AEST Finished 2026-10-08 20:36 AEST (10.7 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab ! kit sweep ! TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T063235Z-lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 55g --memory-swap 55g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-55g-rtx-3090-24gb.json sha256 decb3fcf91f1fe074edadb0343d78d84b6ee565894ff405c0b3bf57540398512MTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 193 tok/s {"prompt_tokens": [10795, 10759, 10750], "ttft_s": [56.03, 50.889, 64.019]}ttft_prefill_32k_tps 278 tok/s {"prompt_tokens": [42966, 42895, 42841], "ttft_s": [166.643, 154.018, 153.111]}ttft_prefill_tps 193 tok/s {"prompt_tokens": [10795, 10759, 10750], "ttft_s": [56.03, 50.889, 64.019]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 4188a3d929072e112d5fc4c196b1f349c8a4f0f8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T102534Z-glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2-ttft.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
20261008T103635Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-load ✓ pass load T42 eligibility: none
Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 Started 2026-10-08 20:36 AEST Finished 2026-10-08 20:38 AEST (1.9 min) Exit status None Notes –
Other runs of this config: load lab ! kit sweep TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29cMTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 113 s host_ram_drop_gb 88.8 GB vram_ready_mib 22,902 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 6d451232712d567aa4886576429949cd7a6729bf dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T103635Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-load.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
20261008T103828Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-lab ! fail lab T42 eligibility: none failed stage: gates
Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 Started 2026-10-08 20:38 AEST Finished 2026-10-08 20:52 AEST (14.3 min) Exit status 0 Notes served=glm-5.3-flash
Other runs of this config: load lab ! kit sweep TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29cMTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max 93,437 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": false}lab_decode_c1_tps 11.3 tok/s lab_prefill_tps 630 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 6d451232712d567aa4886576429949cd7a6729bf dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T103828Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-lab.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
20261008T105246Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-kit_sweep ✓ pass kit sweep T42 eligibility: none
Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 Started 2026-10-08 20:52 AEST Finished 2026-10-08 22:23 AEST (91.1 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load lab ! kit sweep TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29cMTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 379 tok/s {"min": 360.7, "max": 381.4, "n": 3}prefill_16k_tps 520 tok/s {"min": 513.0, "max": 520.5, "n": 3}prefill_32k_tps 621 tok/s {"min": 545.1, "max": 633.6, "n": 3}prefill_64k_tps 676 tok/s {"min": 667.4, "max": 677.6, "n": 3}decode_c1_tps 10.9 tok/s {"min": 10.5, "max": 11.39, "power_w": null}decode_c2_tps 11.0 tok/s {"min": 10.57, "max": 11.41, "power_w": null}decode_c3_tps 10.9 tok/s {"min": 10.53, "max": 11.28, "power_w": null}decode_c4_tps 10.5 tok/s {"min": 10.32, "max": 10.74, "power_w": null}decode_c1_32k_tps 10.5 tok/s {"min": 9.6, "max": 11.36, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 6d451232712d567aa4886576429949cd7a6729bf dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T105246Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-kit_sweep.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
20261008T122352Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-quality_panel ✓ pass quality panel T42 eligibility: none
Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 Started 2026-10-08 22:23 AEST Finished 2026-10-08 22:26 AEST (2.3 min) Exit status 0 Notes panel=glm-5.3-flash-exl3-ref-panel.json rc=0
Other runs of this config: load lab ! kit sweep TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29cMTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run top1_agreement 1.00 fraction mean_kl 0 nats in_band yes "top1>=0.99, KL<=0.001"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 6d451232712d567aa4886576429949cd7a6729bf dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T122352Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-quality_panel.json
Runs / GLM-5.3-Flash / glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2
20261008T122612Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-ttft ✓ pass TTFT T42 eligibility: none
Config glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 Started 2026-10-08 22:26 AEST Finished 2026-10-08 22:31 AEST (5.3 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab ! kit sweep TTFT quality panel
Launch recorded by tools/run.py (artifacts/runs/20261008T103635Z-lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 1g --memory 90g --memory-swap 90g --ulimit memlock=-1 --cpuset-cpus 1,3-16,19-31 -e PORT=30000 -e GLM53_MODE=nvme -e GLM53_EC_MAX_SLOTS=1376 -e GLM53_MAIN_CPUS=1 -e GLM53_NV_CTL_CPU=16 -e GLM53_NV_CPU_CPUS=3-15 -e GLM53_NV_READER_CPUS=19-31 -e HF_HUB_OFFLINE=1 -v ~/models/local-ai-exp/turboderp-GLM-5.3-Flash-exl3-3.05bpw:/models:ro -v ~/models/local-ai-exp/nvx-glm53-v4.1:/nvx:ro ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16 Image ghcr.io/sybil-solutions/glm53-flash-offload@sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Digest sha256:99b8926e01480b1acc35251a60f61cba6a2ad6309491a6ab2bb6b5688f843c16Engine exllamav3 Engine source https://github.com/0xSero/glm53-flash-offload/tree/7f1ee89 Launch file launches/glm53-offload-v4.1-nvme-90g-rtx-3090-24gb.json sha256 6952b4d74789c9d18463812a52173d024806e5cfe0ec985fdb8b37db8d68c29cMTP / speculative off argv env PORT=30000GLM53_MODE=nvmeGLM53_EC_MAX_SLOTS=1376GLM53_MAIN_CPUS=1GLM53_NV_CTL_CPU=16GLM53_NV_CPU_CPUS=3-15GLM53_NV_READER_CPUS=19-31HF_HUB_OFFLINE=1
turboderp/GLM-5.3-Flash-exl3 @332ab457b709b7ba30dd9a448be5de03b80a7ac9 · manifest models/turboderp-GLM-5.3-Flash-exl3-3.05bpw.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 336 tok/s {"prompt_tokens": [10660, 10631, 10711], "ttft_s": [31.688, 31.731, 31.792]}ttft_prefill_32k_tps 576 tok/s {"prompt_tokens": [42848, 42969, 42779], "ttft_s": [74.448, 80.22, 69.643]}ttft_prefill_tps 336 tok/s {"prompt_tokens": [10660, 10631, 10711], "ttft_s": [31.688, 31.731, 31.792]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 6d451232712d567aa4886576429949cd7a6729bf dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/glm-5.3-flash/20261008T122612Z-glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
20261008T123143Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T42 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 Started 2026-10-08 22:31 AEST Finished 2026-10-08 22:36 AEST (4.3 min) Exit status None Notes –
Other runs of this config: load lab lab kit sweep stability !
Launch recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24eMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 260 s host_ram_drop_gb 63.6 GB vram_ready_mib 20,818 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T123143Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
20261008T123604Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T42 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 Started 2026-10-08 22:36 AEST Finished 2026-10-08 22:40 AEST (4.0 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep stability !
Launch recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24eMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 55.3 tok/s lab_prefill_tps 2,107 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T123604Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
20261008T124004Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass kit sweep T42 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 Started 2026-10-08 22:40 AEST Finished 2026-10-08 22:55 AEST (15.0 min) Exit status 0 Notes sweep status: DONE
Other runs of this config: load lab lab kit sweep stability !
Launch recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24eMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run prefill_8k_tps 2,279 tok/s {"min": 2259.9, "max": 2291.3, "n": 3}prefill_16k_tps 2,398 tok/s {"min": 2391.4, "max": 2406.1, "n": 3}prefill_32k_tps 2,435 tok/s {"min": 2434.3, "max": 2437.0, "n": 3}prefill_64k_tps 2,422 tok/s {"min": 2421.1, "max": 2426.8, "n": 3}decode_c1_tps 48.5 tok/s {"min": 47.78, "max": 49.25, "power_w": null}decode_c2_tps 53.5 tok/s {"min": 51.55, "max": 55.47, "power_w": null}decode_c3_tps 55.8 tok/s {"min": 53.66, "max": 57.88, "power_w": null}decode_c4_tps 53.9 tok/s {"min": 53.09, "max": 54.79, "power_w": null}decode_c1_32k_tps 49.1 tok/s {"min": 48.45, "max": 49.82, "power_w": null}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T124004Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
20261008T125504Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ! fail stability T42 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 Started 2026-10-08 22:55 AEST Finished 2026-10-08 23:05 AEST (10.6 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 53 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.954 < floor 50.0
Other runs of this config: load lab lab kit sweep stability !
Launch recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24eMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 28,256 tokens context.headroom_min 176,327 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 53 calls {"per_min": 5.3, "requests": 34}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 39.0 tok/s window 1: 42.27 tok/s window 2: 45.01 tok/s window 3: 44.23 tok/s window 4: 39.08 tok/s window 5: 43.13 tok/s window 6: 42.37 tok/s min 39.08 · max 45.01 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 332, "min_at_generation_s": 242.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 43.1 tok/s {"generation_seconds": 451.013, "generated_tokens": 19446.5}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 171.2, "requests": 14, "tool_calls": 22, "completion_tokens": 5004, "occupied_max": 11964}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 428.8, "requests": 21, "tool_calls": 31, "completion_tokens": 14526, "occupied_max": 28256}] listtasks_over_64k [] instance idscompletion_tokens 19,530 tokens {"finish_reasons": {"tool_calls": 34}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 53 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 39.0 tok/s, whole run 43.1 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T125504Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2
20261008T130540Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass lab T42 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 Started 2026-10-08 23:05 AEST Finished 2026-10-08 23:08 AEST (2.9 min) Exit status 0 Notes served=flashnext
Other runs of this config: load lab lab kit sweep stability !
Launch recorded by tools/run.py (artifacts/runs/20261008T123143Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe.json sha256 61cc953b3a41d6c463fe752c00b9c1e8ae81091aadb9504ab9d3c117a90fb24eMTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max 166,667 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 56.1 tok/s lab_prefill_tps 2,222 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2591918bae6ffa3cd0573aee2a3543ae8fa95f3c dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T130540Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T132228Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load ✓ pass load T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:22 AEST Finished 2026-10-08 23:23 AEST (0.7 min) Exit status None Notes –
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 43.0 s host_ram_drop_gb 56.9 GB vram_ready_mib 23,660 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T132228Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T132312Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab ✓ pass lab T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:23 AEST Finished 2026-10-08 23:27 AEST (3.9 min) Exit status 0 Notes served=qwen3.8-flash-next-iq3_s
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 78.1 tok/s lab_prefill_tps 2,174 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T132312Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T132706Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-ttft ✓ pass TTFT T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:27 AEST Finished 2026-10-08 23:28 AEST (1.6 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,036 tok/s {"prompt_tokens": [13659, 13791, 13724], "ttft_s": [6.868, 6.757, 6.742]}ttft_prefill_32k_tps 2,207 tok/s {"prompt_tokens": [54790, 54605, 54677], "ttft_s": [24.701, 24.741, 24.794]}ttft_prefill_tps 2,036 tok/s {"prompt_tokens": [13659, 13791, 13724], "ttft_s": [6.868, 6.757, 6.742]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T132706Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T132842Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-stability ✓ pass stability T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:28 AEST Finished 2026-10-08 23:39 AEST (10.5 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 77 tool calls in this run count toward the config's 100-call minimum (§5)
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 39,624 tokens context.headroom_min 91,304 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 77 calls {"per_min": 7.7, "requests": 54}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 87.3 tok/s window 1: 87.58 tok/s window 2: 88.73 tok/s min 87.58 · max 88.73 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 81, "min_at_generation_s": 61.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 85.9 tok/s {"generation_seconds": 200.654, "generated_tokens": 17228.6}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 1 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 87.4, "requests": 13, "tool_calls": 23, "completion_tokens": 3377, "occupied_max": 8278}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 512.8, "requests": 41, "tool_calls": 54, "completion_tokens": 13916, "occupied_max": 39624}] listtasks_over_64k [] instance idscompletion_tokens 17,293 tokens {"finish_reasons": {"tool_calls": 54}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 77 tool calls, 0 parse failures, 0 repetition hits, tasks solved 1/2, decode min window 87.3 tok/s, whole run 85.9 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T132842Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-stability.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T133915Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab ✓ pass lab T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:39 AEST Finished 2026-10-08 23:42 AEST (3.7 min) Exit status 0 Notes served=qwen3.8-flash-next-iq3_s
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T132228Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 75.5 tok/s lab_prefill_tps 2,178 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1f76c0a2475d11e86cc882ccae1105bca4b63d49 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T133915Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T134305Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load ✓ pass load T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:43 AEST Finished 2026-10-08 23:43 AEST (0.4 min) Exit status None Notes –
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T134305Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 24.9 s host_ram_drop_gb 40.1 GB vram_ready_mib 23,646 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1baa1b719899523007cf4a95b1ee26102443d216 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T134305Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261008T134331Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval ✓ pass quality eval T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-08 23:43 AEST Finished 2026-10-09 01:40 AEST (117.1 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T134305Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 54,986 tokens context.headroom_min 76,006 tokens tasks_attempted 30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 28 tasks {"eval_errors": 0}solve_rate 0.9333 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14826, "budget_exhausted": false, "seconds": 358.6, "requests": 40, "tool_calls": 45, "occupied_max": 34317}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17039, "budget_exhausted": false, "seconds": 285.5, "requests": 45, "tool_calls": 58, "occupied_max": 36419}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7793, "budget_exhausted": false, "seconds": 155.4, "requests": 20, "tool_calls": 33, "occupied_max": 22226}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15480, "budget_exhausted": false, "seconds": 185.4, "requests": 28, "tool_calls": 40, "occupied_max": 35799}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8323, "budget_exhausted": false, "seconds": 116.7, "requests": 34, "tool_calls": 34, "occupied_max": 20937}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21342, "budget_exhausted": false, "seconds": 377.0, "requests": 59, "tool_calls": 70, "occupied_max": 42618}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7168, "budget_exhausted": false, "seconds": 126.2, "requests": 21, "tool_calls": 30, "occupied_max": 19405}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 10805, "budget_exhausted": false, "seconds": 114.1, "requests": 18, "tool_calls": 26, "occupied_max": 23420}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9637, "budget_exhausted": false, "seconds": 110.4, "requests": 21, "tool_calls": 35, "occupied_max": 24324}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7201, "budget_exhausted": false, "seconds": 90.3, "requests": 23, "tool_calls": 28, "occupied_max": 15577}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7092, "budget_exhausted": false, "seconds": 108.7, "requests": 29, "tool_calls": 33, "occupied_max": 19064}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8009, "budget_exhausted": false, "seconds": 97.4, "requests": 21, "tool_calls": 30, "occupied_max": 22409}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8174, "budget_exhausted": false, "seconds": 214.2, "requests": 29, "tool_calls": 39, "occupied_max": 21183}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15628, "budget_exhausted": false, "seconds": 362.7, "requests": 68, "tool_calls": 79, "occupied_max": 50516}, {"instance_id": "sympy__sympy-12171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11275, "budget_exhausted": false, "seconds": 150.2, "requests": 26, "tool_calls": 35, "occupied_max": 23599}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18094, "budget_exhausted": false, "seconds": 244.5, "requests": 63, "tool_calls": 69, "occupied_max": 46398}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6936, "budget_exhausted": false, "seconds": 78.0, "requests": 17, "tool_calls": 21, "occupied_max": 16200}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11000, "budget_exhausted": false, "seconds": 130.3, "requests": 32, "tool_calls": 34, "occupied_max": 23676}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14879, "budget_exhausted": false, "seconds": 252.9, "requests": 45, "tool_calls": 61, "occupied_max": 34306}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25688, "budget_exhausted": false, "seconds": 517.0, "requests": 55, "tool_calls": 60, "occupied_max": 53147}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12224, "budget_exhausted": false, "seconds": 144.9, "requests": 28, "tool_calls": 38, "occupied_max": 30693}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18211, "budget_exhausted": false, "seconds": 354.9, "requests": 59, "tool_calls": 65, "occupied_max": 46730}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15845, "budget_exhausted": false, "seconds": 273.4, "requests": 41, "tool_calls": 56, "occupied_max": 45996}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29631, "budget_exhausted": false, "seconds": 461.1, "requests": 56, "tool_calls": 63, "occupied_max": 54986}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 1841, "budget_exhausted": false, "seconds": 30.5, "requests": 11, "tool_calls": 14, "occupied_max": 6242}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11918, "budget_exhausted": false, "seconds": 241.8, "requests": 36, "tool_calls": 40, "occupied_max": 34082}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7487, "budget_exhausted": false, "seconds": 93.2, "requests": 30, "tool_calls": 33, "occupied_max": 20098}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 24675, "budget_exhausted": false, "seconds": 448.4, "requests": 48, "tool_calls": 58, "occupied_max": 50684}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22047, "budget_exhausted": false, "seconds": 240.8, "requests": 39, "tool_calls": 46, "occupied_max": 43171}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5504, "budget_exhausted": false, "seconds": 89.3, "requests": 23, "tool_calls": 25, "occupied_max": 15117}] listtool_calls 1,298 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 0 responses {"wire": {}, "harness_format_errors": {"no_tool_call": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 1 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 5 hits {"ngram64x3": 5, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}decode_whole_run_tps 126 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 5a6b28c5783accaacc1c2f0e2472f03355d9caa4 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T134331Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T162803Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load ✓ pass load T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-09 02:28 AEST Finished 2026-10-09 02:28 AEST (0.5 min) Exit status None Notes –
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T162803Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 30.9 s host_ram_drop_gb 56.8 GB vram_ready_mib 23,660 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 5306e4acb0da904fb556961094c6256c16b09f38 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T162803Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261008T162835Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval ✓ pass quality eval T41 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-09 02:28 AEST Finished 2026-10-09 05:40 AEST (192.3 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261008T162803Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 60,497 tokens context.headroom_min 70,102 tokens tasks_attempted 30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 29 tasks {"eval_errors": 0}solve_rate 0.9667 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10130, "budget_exhausted": false, "seconds": 547.2, "requests": 32, "tool_calls": 49, "occupied_max": 26721}, {"instance_id": "sympy__sympy-20442", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9949, "budget_exhausted": false, "seconds": 258.1, "requests": 23, "tool_calls": 32, "occupied_max": 22571}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13391, "budget_exhausted": false, "seconds": 353.5, "requests": 35, "tool_calls": 49, "occupied_max": 30447}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20221, "budget_exhausted": false, "seconds": 512.2, "requests": 33, "tool_calls": 43, "occupied_max": 54956}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9804, "budget_exhausted": false, "seconds": 183.3, "requests": 24, "tool_calls": 28, "occupied_max": 22437}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21892, "budget_exhausted": false, "seconds": 551.5, "requests": 34, "tool_calls": 51, "occupied_max": 37587}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9860, "budget_exhausted": false, "seconds": 230.6, "requests": 24, "tool_calls": 38, "occupied_max": 29374}, {"instance_id": "sympy__sympy-18199", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 17676, "budget_exhausted": false, "seconds": 424.8, "requests": 27, "tool_calls": 39, "occupied_max": 32315}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7213, "budget_exhausted": false, "seconds": 131.6, "requests": 20, "tool_calls": 33, "occupied_max": 17051}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10879, "budget_exhausted": false, "seconds": 384.1, "requests": 34, "tool_calls": 47, "occupied_max": 23162}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8532, "budget_exhausted": false, "seconds": 166.0, "requests": 20, "tool_calls": 32, "occupied_max": 17827}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11580, "budget_exhausted": false, "seconds": 268.9, "requests": 30, "tool_calls": 49, "occupied_max": 31286}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10747, "budget_exhausted": false, "seconds": 596.1, "requests": 30, "tool_calls": 50, "occupied_max": 25175}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16507, "budget_exhausted": false, "seconds": 430.6, "requests": 52, "tool_calls": 68, "occupied_max": 55285}, {"instance_id": "sympy__sympy-12171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26317, "budget_exhausted": false, "seconds": 415.9, "requests": 39, "tool_calls": 55, "occupied_max": 49621}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14483, "budget_exhausted": false, "seconds": 568.5, "requests": 56, "tool_calls": 66, "occupied_max": 33074}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7734, "budget_exhausted": false, "seconds": 137.6, "requests": 18, "tool_calls": 24, "occupied_max": 17099}, {"instance_id": "django__django-16873", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10270, "budget_exhausted": false, "seconds": 186.6, "requests": 24, "tool_calls": 37, "occupied_max": 20073}, {"instance_id": "astropy__astropy-6938", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16593, "budget_exhausted": false, "seconds": 356.3, "requests": 46, "tool_calls": 63, "occupied_max": 36246}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17970, "budget_exhausted": false, "seconds": 396.3, "requests": 40, "tool_calls": 56, "occupied_max": 44655}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15975, "budget_exhausted": false, "seconds": 350.9, "requests": 31, "tool_calls": 42, "occupied_max": 35284}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28980, "budget_exhausted": false, "seconds": 627.0, "requests": 63, "tool_calls": 88, "occupied_max": 56435}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27648, "budget_exhausted": false, "seconds": 775.0, "requests": 55, "tool_calls": 83, "occupied_max": 60497}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16028, "budget_exhausted": false, "seconds": 331.5, "requests": 32, "tool_calls": 36, "occupied_max": 28287}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6555, "budget_exhausted": false, "seconds": 133.6, "requests": 17, "tool_calls": 29, "occupied_max": 15328}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19450, "budget_exhausted": false, "seconds": 512.0, "requests": 32, "tool_calls": 50, "occupied_max": 41907}, {"instance_id": "django__django-14672", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4607, "budget_exhausted": false, "seconds": 96.0, "requests": 18, "tool_calls": 24, "occupied_max": 13103}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15280, "budget_exhausted": false, "seconds": 374.1, "requests": 25, "tool_calls": 40, "occupied_max": 32560}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12348, "budget_exhausted": false, "seconds": 353.7, "requests": 32, "tool_calls": 41, "occupied_max": 30299}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12660, "budget_exhausted": false, "seconds": 324.1, "requests": 25, "tool_calls": 37, "occupied_max": 24670}] listtool_calls 1,379 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 3 hits {"ngram64x3": 3, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}decode_whole_run_tps 83.5 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 619eb8d354d7d6dbb4bf0039ff6c616d5388cbf2 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261008T162835Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261009T010134Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load ✓ pass load T43 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-09 11:01 AEST Finished 2026-10-09 11:02 AEST (0.5 min) Exit status None Notes –
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261009T010134Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 31.0 s host_ram_drop_gb 56.6 GB vram_ready_mib 23,660 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 901f805673b7fb363b4434146981d5e5dfa5679f dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261009T010134Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261009T010206Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval ✓ pass quality eval T43 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-09 11:02 AEST Finished 2026-10-09 21:36 AEST (633.9 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261009T010134Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 123,185 tokens context.headroom_min 7,250 tokens tasks_attempted 50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 48 tasks {"eval_errors": 1}solve_rate 0.96 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34142, "budget_exhausted": false, "seconds": 795.0, "requests": 75, "tool_calls": 91, "occupied_max": 88574}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40034, "budget_exhausted": false, "seconds": 901.5, "requests": 53, "tool_calls": 66, "occupied_max": 81138}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21957, "budget_exhausted": false, "seconds": 509.4, "requests": 52, "tool_calls": 74, "occupied_max": 59240}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7793, "budget_exhausted": false, "seconds": 124.8, "requests": 13, "tool_calls": 23, "occupied_max": 17264}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17072, "budget_exhausted": false, "seconds": 607.7, "requests": 47, "tool_calls": 69, "occupied_max": 49060}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 46730, "budget_exhausted": false, "seconds": 1276.2, "requests": 87, "tool_calls": 89, "occupied_max": 89048}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18601, "budget_exhausted": false, "seconds": 506.9, "requests": 48, "tool_calls": 63, "occupied_max": 48573}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18441, "budget_exhausted": false, "seconds": 460.1, "requests": 41, "tool_calls": 59, "occupied_max": 41645}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 35736, "budget_exhausted": false, "seconds": 811.1, "requests": 56, "tool_calls": 77, "occupied_max": 70836}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20548, "budget_exhausted": false, "seconds": 352.9, "requests": 24, "tool_calls": 41, "occupied_max": 44792}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17759, "budget_exhausted": false, "seconds": 407.5, "requests": 38, "tool_calls": 47, "occupied_max": 37704}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 41340, "budget_exhausted": false, "seconds": 597.9, "requests": 20, "tool_calls": 29, "occupied_max": 61052}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20349, "budget_exhausted": false, "seconds": 613.8, "requests": 32, "tool_calls": 53, "occupied_max": 41785}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 36273, "budget_exhausted": false, "seconds": 822.7, "requests": 60, "tool_calls": 80, "occupied_max": 77817}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 11846, "budget_exhausted": false, "seconds": 333.6, "requests": 40, "tool_calls": 40, "occupied_max": 34587}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8182, "budget_exhausted": false, "seconds": 221.9, "requests": 14, "tool_calls": 20, "occupied_max": 18033}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18733, "budget_exhausted": false, "seconds": 1487.2, "requests": 58, "tool_calls": 79, "occupied_max": 47859}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12272, "budget_exhausted": false, "seconds": 315.3, "requests": 27, "tool_calls": 38, "occupied_max": 34431}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22608, "budget_exhausted": false, "seconds": 405.7, "requests": 42, "tool_calls": 42, "occupied_max": 44932}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14068, "budget_exhausted": false, "seconds": 264.0, "requests": 28, "tool_calls": 46, "occupied_max": 31724}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40790, "budget_exhausted": false, "seconds": 1774.3, "requests": 95, "tool_calls": 116, "occupied_max": 99465}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 46281, "budget_exhausted": false, "seconds": 1317.5, "requests": 70, "tool_calls": 90, "occupied_max": 85084}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 23745, "budget_exhausted": false, "seconds": 395.1, "requests": 39, "tool_calls": 57, "occupied_max": 46650}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18612, "budget_exhausted": false, "seconds": 495.4, "requests": 40, "tool_calls": 59, "occupied_max": 50057}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18608, "budget_exhausted": false, "seconds": 731.9, "requests": 59, "tool_calls": 78, "occupied_max": 51060}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 45225, "budget_exhausted": false, "seconds": 827.7, "requests": 77, "tool_calls": 108, "occupied_max": 94483}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40896, "budget_exhausted": false, "seconds": 995.6, "requests": 70, "tool_calls": 82, "occupied_max": 77970}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34432, "budget_exhausted": false, "seconds": 1068.6, "requests": 54, "tool_calls": 94, "occupied_max": 74341}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19589, "budget_exhausted": false, "seconds": 337.3, "requests": 39, "tool_calls": 54, "occupied_max": 44753}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6831, "budget_exhausted": false, "seconds": 301.8, "requests": 25, "tool_calls": 43, "occupied_max": 21257}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37119, "budget_exhausted": false, "seconds": 2290.1, "requests": 110, "tool_calls": 146, "occupied_max": 97799}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34047, "budget_exhausted": false, "seconds": 964.8, "requests": 70, "tool_calls": 92, "occupied_max": 78892}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 54771, "budget_exhausted": false, "seconds": 1772.6, "requests": 101, "tool_calls": 103, "occupied_max": 123185}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10761, "budget_exhausted": false, "seconds": 365.1, "requests": 53, "tool_calls": 73, "occupied_max": 34663}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19170, "budget_exhausted": false, "seconds": 480.9, "requests": 46, "tool_calls": 61, "occupied_max": 51197}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18401, "budget_exhausted": false, "seconds": 385.6, "requests": 36, "tool_calls": 56, "occupied_max": 44456}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28022, "budget_exhausted": false, "seconds": 548.1, "requests": 50, "tool_calls": 74, "occupied_max": 66084}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20474, "budget_exhausted": false, "seconds": 1089.8, "requests": 55, "tool_calls": 80, "occupied_max": 52817}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16898, "budget_exhausted": false, "seconds": 542.9, "requests": 27, "tool_calls": 45, "occupied_max": 32301}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13069, "budget_exhausted": false, "seconds": 728.4, "requests": 34, "tool_calls": 50, "occupied_max": 34588}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29334, "budget_exhausted": false, "seconds": 745.8, "requests": 90, "tool_calls": 123, "occupied_max": 72326}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22950, "budget_exhausted": false, "seconds": 789.3, "requests": 40, "tool_calls": 54, "occupied_max": 42461}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14617, "budget_exhausted": false, "seconds": 331.1, "requests": 28, "tool_calls": 49, "occupied_max": 47256}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10457, "budget_exhausted": false, "seconds": 178.2, "requests": 17, "tool_calls": 26, "occupied_max": 23948}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31763, "budget_exhausted": false, "seconds": 783.2, "requests": 65, "tool_calls": 75, "occupied_max": 69969}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 41002, "budget_exhausted": false, "seconds": 1626.9, "requests": 83, "tool_calls": 107, "occupied_max": 107504}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24821, "budget_exhausted": false, "seconds": 1062.5, "requests": 69, "tool_calls": 80, "occupied_max": 64259}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12706, "budget_exhausted": false, "seconds": 265.4, "requests": 23, "tool_calls": 37, "occupied_max": 35352}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 50296, "budget_exhausted": false, "seconds": 1224.1, "requests": 71, "tool_calls": 94, "occupied_max": 91537}, {"instance_id": "roboflow__supervision-2600", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19271, "budget_exhausted": false, "seconds": 414.4, "requests": 24, "tool_calls": 34, "occupied_max": 41350}] listtool_calls 3,366 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 1 responses {"wire": {}, "harness_format_errors": {"truncated_no_tool_call": 1, "missing_command": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 1 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 6 hits {"ngram64x3": 6, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}decode_whole_run_tps 83.0 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 2af53e6e0ae8bbe0782719da7dfce3c7b95379b5 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261009T010206Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-quality_eval.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261009T113616Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo ✓ pass load T43 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-09 21:36 AEST Finished 2026-10-09 21:38 AEST (2.2 min) Exit status None Notes –
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261009T113616Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 131 s host_ram_drop_gb 7.77 GB vram_ready_mib 22,372 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 678111cd311988184237eefaeef779f3344d0893 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261009T113616Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261009T113828Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-qu ✓ pass quality eval T43 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-09 21:38 AEST Finished 2026-10-10 03:48 AEST (369.6 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261009T113616Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max 86,226 tokens context.headroom_min 112,429 tokens tasks_attempted 50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 36 tasks {"eval_errors": 1}solve_rate 0.72 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 45078, "budget_exhausted": false, "seconds": 657.7, "requests": 73, "tool_calls": 96, "occupied_max": 85404}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 22559, "budget_exhausted": false, "seconds": 426.0, "requests": 46, "tool_calls": 60, "occupied_max": 57351}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26432, "budget_exhausted": false, "seconds": 456.1, "requests": 84, "tool_calls": 94, "occupied_max": 66786}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9552, "budget_exhausted": false, "seconds": 93.7, "requests": 18, "tool_calls": 25, "occupied_max": 20127}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13054, "budget_exhausted": false, "seconds": 277.6, "requests": 38, "tool_calls": 54, "occupied_max": 38748}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 23547, "budget_exhausted": false, "seconds": 561.3, "requests": 46, "tool_calls": 57, "occupied_max": 62365}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13663, "budget_exhausted": false, "seconds": 257.6, "requests": 33, "tool_calls": 45, "occupied_max": 31906}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21920, "budget_exhausted": false, "seconds": 255.6, "requests": 38, "tool_calls": 52, "occupied_max": 37867}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 54531, "budget_exhausted": false, "seconds": 576.0, "requests": 72, "tool_calls": 81, "occupied_max": 76756}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22406, "budget_exhausted": false, "seconds": 244.2, "requests": 31, "tool_calls": 34, "occupied_max": 43755}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14980, "budget_exhausted": false, "seconds": 220.6, "requests": 27, "tool_calls": 30, "occupied_max": 27766}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 55860, "budget_exhausted": false, "seconds": 486.1, "requests": 14, "tool_calls": 10, "occupied_max": 16731}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21555, "budget_exhausted": false, "seconds": 354.8, "requests": 26, "tool_calls": 30, "occupied_max": 28568}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12669, "budget_exhausted": false, "seconds": 170.3, "requests": 34, "tool_calls": 37, "occupied_max": 26755}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 8560, "budget_exhausted": false, "seconds": 193.3, "requests": 24, "tool_calls": 30, "occupied_max": 28471}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22798, "budget_exhausted": false, "seconds": 290.5, "requests": 17, "tool_calls": 23, "occupied_max": 25055}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16220, "budget_exhausted": false, "seconds": 669.1, "requests": 41, "tool_calls": 60, "occupied_max": 41671}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8340, "budget_exhausted": false, "seconds": 477.3, "requests": 35, "tool_calls": 41, "occupied_max": 22159}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 25440, "budget_exhausted": false, "seconds": 288.1, "requests": 37, "tool_calls": 44, "occupied_max": 33085}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24466, "budget_exhausted": false, "seconds": 259.4, "requests": 35, "tool_calls": 50, "occupied_max": 48402}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 42568, "budget_exhausted": false, "seconds": 764.5, "requests": 54, "tool_calls": 61, "occupied_max": 66917}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 627.5, "requests": 56, "tool_calls": 60, "occupied_max": 71198}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 28160, "budget_exhausted": false, "seconds": 245.2, "requests": 25, "tool_calls": 29, "occupied_max": 26916}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11714, "budget_exhausted": false, "seconds": 140.8, "requests": 34, "tool_calls": 40, "occupied_max": 28498}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11519, "budget_exhausted": false, "seconds": 165.3, "requests": 30, "tool_calls": 36, "occupied_max": 27020}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18428, "budget_exhausted": false, "seconds": 244.8, "requests": 49, "tool_calls": 59, "occupied_max": 54832}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 715.7, "requests": 53, "tool_calls": 59, "occupied_max": 82453}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33017, "budget_exhausted": false, "seconds": 418.2, "requests": 46, "tool_calls": 65, "occupied_max": 66120}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 34551, "budget_exhausted": false, "seconds": 345.6, "requests": 67, "tool_calls": 90, "occupied_max": 55591}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5124, "budget_exhausted": false, "seconds": 196.1, "requests": 22, "tool_calls": 33, "occupied_max": 14181}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 2250.1, "requests": 121, "tool_calls": 135, "occupied_max": 82191}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 49347, "budget_exhausted": false, "seconds": 699.2, "requests": 69, "tool_calls": 77, "occupied_max": 76656}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 685.2, "requests": 59, "tool_calls": 98, "occupied_max": 85573}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4433, "budget_exhausted": false, "seconds": 91.7, "requests": 21, "tool_calls": 33, "occupied_max": 16310}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15980, "budget_exhausted": false, "seconds": 222.4, "requests": 43, "tool_calls": 67, "occupied_max": 38851}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18842, "budget_exhausted": false, "seconds": 260.3, "requests": 48, "tool_calls": 65, "occupied_max": 45432}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33864, "budget_exhausted": false, "seconds": 423.4, "requests": 51, "tool_calls": 60, "occupied_max": 56131}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19756, "budget_exhausted": false, "seconds": 613.8, "requests": 55, "tool_calls": 60, "occupied_max": 49105}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15454, "budget_exhausted": false, "seconds": 204.7, "requests": 29, "tool_calls": 38, "occupied_max": 29449}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16226, "budget_exhausted": false, "seconds": 237.5, "requests": 27, "tool_calls": 32, "occupied_max": 36979}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 30651, "budget_exhausted": false, "seconds": 353.4, "requests": 67, "tool_calls": 89, "occupied_max": 61003}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18276, "budget_exhausted": false, "seconds": 403.8, "requests": 48, "tool_calls": 50, "occupied_max": 33827}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8573, "budget_exhausted": false, "seconds": 151.7, "requests": 30, "tool_calls": 43, "occupied_max": 28911}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19383, "budget_exhausted": false, "seconds": 191.7, "requests": 17, "tool_calls": 25, "occupied_max": 31319}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 57933, "budget_exhausted": false, "seconds": 729.1, "requests": 55, "tool_calls": 57, "occupied_max": 68084}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37411, "budget_exhausted": false, "seconds": 648.6, "requests": 77, "tool_calls": 96, "occupied_max": 75095}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 514.9, "requests": 41, "tool_calls": 42, "occupied_max": 35461}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12409, "budget_exhausted": false, "seconds": 146.4, "requests": 28, "tool_calls": 39, "occupied_max": 29397}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 49593, "budget_exhausted": false, "seconds": 773.7, "requests": 59, "tool_calls": 73, "occupied_max": 86226}, {"instance_id": "roboflow__supervision-2600", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": false, "seconds": 596.9, "requests": 19, "tool_calls": 18, "occupied_max": 45787}] listtool_calls 2,682 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 4 responses {"wire": {"invalid_tool_call": 3}, "harness_format_errors": {"truncated_no_tool_call": 39, "no_tool_call": 3, "unknown_tool": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 39 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 3 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 67 hits {"ngram64x3": 67, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}decode_whole_run_tps 119 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records mtp_acceptance_rate 0.7123 fraction {"mean_accept_len": 3.137, "batches": 11337, "source": "SGLang decode log"}mtp_accepted_tps 119 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 678111cd311988184237eefaeef779f3344d0893 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261009T113828Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-qu.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261009T174823Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load ✓ pass load T43 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-10 03:48 AEST Finished 2026-10-10 03:48 AEST (0.4 min) Exit status None Notes –
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261009T174823Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 24.9 s host_ram_drop_gb 40.1 GB vram_ready_mib 23,646 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e329e6c5aaddf023f1cdd7b407bd8266a2387991 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261009T174823Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261009T174849Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval ✓ pass quality eval T43 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-10 03:48 AEST Finished 2026-10-10 10:22 AEST (393.9 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261009T174823Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 125,121 tokens context.headroom_min 5,323 tokens tasks_attempted 50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 46 tasks {"eval_errors": 1}solve_rate 0.92 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 44538, "budget_exhausted": false, "seconds": 652.8, "requests": 84, "tool_calls": 94, "occupied_max": 97150}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24420, "budget_exhausted": false, "seconds": 488.3, "requests": 51, "tool_calls": 58, "occupied_max": 59275}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18066, "budget_exhausted": false, "seconds": 287.3, "requests": 59, "tool_calls": 68, "occupied_max": 49652}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 5949, "budget_exhausted": false, "seconds": 60.0, "requests": 11, "tool_calls": 16, "occupied_max": 13786}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11418, "budget_exhausted": false, "seconds": 293.8, "requests": 47, "tool_calls": 58, "occupied_max": 39569}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31571, "budget_exhausted": false, "seconds": 832.0, "requests": 69, "tool_calls": 75, "occupied_max": 69549}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9136, "budget_exhausted": false, "seconds": 162.1, "requests": 27, "tool_calls": 35, "occupied_max": 30791}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12924, "budget_exhausted": false, "seconds": 268.6, "requests": 31, "tool_calls": 41, "occupied_max": 31169}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 21953, "budget_exhausted": false, "seconds": 267.6, "requests": 42, "tool_calls": 48, "occupied_max": 47328}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17697, "budget_exhausted": false, "seconds": 212.0, "requests": 39, "tool_calls": 45, "occupied_max": 40561}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18657, "budget_exhausted": false, "seconds": 262.2, "requests": 23, "tool_calls": 27, "occupied_max": 32044}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 47905, "budget_exhausted": false, "seconds": 553.2, "requests": 34, "tool_calls": 36, "occupied_max": 57301}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9749, "budget_exhausted": false, "seconds": 183.1, "requests": 23, "tool_calls": 24, "occupied_max": 22993}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27497, "budget_exhausted": false, "seconds": 479.9, "requests": 47, "tool_calls": 57, "occupied_max": 54933}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 9342, "budget_exhausted": false, "seconds": 217.1, "requests": 29, "tool_calls": 37, "occupied_max": 31459}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8072, "budget_exhausted": false, "seconds": 106.0, "requests": 12, "tool_calls": 17, "occupied_max": 17177}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10249, "budget_exhausted": false, "seconds": 1395.2, "requests": 47, "tool_calls": 56, "occupied_max": 35318}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4615, "budget_exhausted": false, "seconds": 108.1, "requests": 17, "tool_calls": 21, "occupied_max": 18705}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 12411, "budget_exhausted": false, "seconds": 173.7, "requests": 28, "tool_calls": 34, "occupied_max": 28579}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14133, "budget_exhausted": false, "seconds": 167.8, "requests": 26, "tool_calls": 35, "occupied_max": 33597}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 51570, "budget_exhausted": false, "seconds": 988.2, "requests": 88, "tool_calls": 102, "occupied_max": 113525}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 41744, "budget_exhausted": false, "seconds": 685.9, "requests": 65, "tool_calls": 73, "occupied_max": 80846}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16370, "budget_exhausted": false, "seconds": 196.6, "requests": 43, "tool_calls": 45, "occupied_max": 33487}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15399, "budget_exhausted": false, "seconds": 198.4, "requests": 34, "tool_calls": 50, "occupied_max": 44161}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11607, "budget_exhausted": false, "seconds": 226.1, "requests": 42, "tool_calls": 51, "occupied_max": 28478}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29071, "budget_exhausted": false, "seconds": 400.3, "requests": 63, "tool_calls": 75, "occupied_max": 73574}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 669.9, "requests": 58, "tool_calls": 63, "occupied_max": 110093}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26511, "budget_exhausted": false, "seconds": 436.5, "requests": 65, "tool_calls": 90, "occupied_max": 71966}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31086, "budget_exhausted": false, "seconds": 377.5, "requests": 77, "tool_calls": 87, "occupied_max": 71838}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9729, "budget_exhausted": false, "seconds": 202.4, "requests": 33, "tool_calls": 44, "occupied_max": 25649}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33639, "budget_exhausted": false, "seconds": 2630.1, "requests": 129, "tool_calls": 143, "occupied_max": 81449}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28295, "budget_exhausted": false, "seconds": 764.2, "requests": 99, "tool_calls": 119, "occupied_max": 75551}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1429.1, "requests": 106, "tool_calls": 112, "occupied_max": 125121}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 3606, "budget_exhausted": false, "seconds": 80.0, "requests": 21, "tool_calls": 28, "occupied_max": 14274}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14623, "budget_exhausted": false, "seconds": 211.1, "requests": 35, "tool_calls": 45, "occupied_max": 40292}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9638, "budget_exhausted": false, "seconds": 133.2, "requests": 26, "tool_calls": 38, "occupied_max": 31805}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26209, "budget_exhausted": false, "seconds": 318.9, "requests": 38, "tool_calls": 49, "occupied_max": 54186}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18365, "budget_exhausted": false, "seconds": 572.4, "requests": 39, "tool_calls": 59, "occupied_max": 56748}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16460, "budget_exhausted": false, "seconds": 427.6, "requests": 35, "tool_calls": 40, "occupied_max": 30798}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12223, "budget_exhausted": false, "seconds": 260.6, "requests": 30, "tool_calls": 40, "occupied_max": 34837}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16756, "budget_exhausted": false, "seconds": 363.2, "requests": 52, "tool_calls": 69, "occupied_max": 46110}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12158, "budget_exhausted": false, "seconds": 385.1, "requests": 30, "tool_calls": 33, "occupied_max": 22195}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8741, "budget_exhausted": false, "seconds": 210.8, "requests": 30, "tool_calls": 39, "occupied_max": 29740}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8337, "budget_exhausted": false, "seconds": 96.0, "requests": 22, "tool_calls": 22, "occupied_max": 18404}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37422, "budget_exhausted": false, "seconds": 495.7, "requests": 56, "tool_calls": 58, "occupied_max": 70022}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31749, "budget_exhausted": false, "seconds": 1069.9, "requests": 76, "tool_calls": 84, "occupied_max": 80814}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14637, "budget_exhausted": false, "seconds": 307.2, "requests": 48, "tool_calls": 54, "occupied_max": 44288}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6244, "budget_exhausted": false, "seconds": 93.8, "requests": 21, "tool_calls": 26, "occupied_max": 20934}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34275, "budget_exhausted": false, "seconds": 734.5, "requests": 71, "tool_calls": 96, "occupied_max": 68587}, {"instance_id": "roboflow__supervision-2600", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22526, "budget_exhausted": false, "seconds": 297.8, "requests": 28, "tool_calls": 32, "occupied_max": 42258}] listtool_calls 2,748 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 0 responses {"wire": {}, "harness_format_errors": {"truncated_no_tool_call": 3, "no_tool_call": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 3 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 1 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 7 hits {"ngram64x3": 6, "same_call_x4": 1, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}decode_whole_run_tps 123 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e329e6c5aaddf023f1cdd7b407bd8266a2387991 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261009T174849Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-quality_eval.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261010T002303Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T43 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-10 10:23 AEST Finished 2026-10-10 10:27 AEST (4.4 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261010T002303Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 267 s host_ram_drop_gb 65.1 GB vram_ready_mib 20,844 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 29c8bc5c793d774d8c9cb4d73dba668b9d156223 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T002303Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261010T104600Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load ✓ pass load T44 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-10 20:46 AEST Finished 2026-10-10 20:46 AEST (0.4 min) Exit status None Notes –
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261010T104600Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 25.1 s host_ram_drop_gb 40.0 GB vram_ready_mib 23,646 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit ca980ac49e0fa5fad31e9e3a3b3c32c4ec092e7e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T104600Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2
20261010T104626Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-resource_profile ✓ pass resources T44 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 Started 2026-10-10 20:46 AEST Finished 2026-10-10 20:48 AEST (1.8 min) Exit status 0 Notes resource profile; GPU index 2; probe 1065 samples
Other runs of this config: load load load load lab lab kit sweep ! TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261010T104600Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gp --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run idle_vram_used_gib_max 23.5 GiB idle_cpu_cores_busy_mean 0.1 cores idle_cpu_cores_busy_p95 0.27 cores idle_gpu_util_mean_pct 0 % idle_pcie_rx_gbps_mean 0 GB/s idle_power_w_mean 100 W idle_ram_anon_gib 33.2 GiB "container cgroup memory.stat at phase end"idle_ram_file_gib 39.1 GiB "container cgroup memory.stat at phase end"prefill_vram_used_gib_max 23.6 GiB prefill_cpu_cores_busy_mean 1.33 cores prefill_cpu_cores_busy_p95 2.88 cores prefill_gpu_util_mean_pct 93.4 % prefill_pcie_rx_gbps_mean 8.42 GB/s prefill_power_w_mean 348 W prefill_ram_anon_gib 33.6 GiB "container cgroup memory.stat at phase end"prefill_ram_file_gib 39.1 GiB "container cgroup memory.stat at phase end"decode_vram_used_gib_max 23.6 GiB decode_cpu_cores_busy_mean 15.2 cores decode_cpu_cores_busy_p95 15.6 cores decode_gpu_util_mean_pct 99.6 % decode_pcie_rx_gbps_mean 0.78 GB/s decode_power_w_mean 381 W decode_ram_anon_gib 33.6 GiB "container cgroup memory.stat at phase end"decode_ram_file_gib 39.1 GiB "container cgroup memory.stat at phase end"prefill_tps_median 2,584 tok/s {"prompt_tokens": [26958, 27045, 27018]}decode_tps_median 123 tok/s {"completion_tokens": [2007, 2047, 2048]}
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit ca980ac49e0fa5fad31e9e3a3b3c32c4ec092e7e dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T104626Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2-resource_profile.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261010T104819Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load ✓ pass load T44 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-10 20:48 AEST Finished 2026-10-10 20:48 AEST (0.5 min) Exit status None Notes –
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261010T104819Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 31.1 s host_ram_drop_gb 56.9 GB vram_ready_mib 23,660 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit dbbb8ed9b3962d9817eca755993dbac443c161e8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T104819Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2
20261010T104851Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-resource_profile ✓ pass resources T44 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2 Started 2026-10-10 20:48 AEST Finished 2026-10-10 20:50 AEST (2.1 min) Exit status 0 Notes resource profile; GPU index 2; probe 1264 samples
Other runs of this config: load load load load lab lab TTFT stability quality eval quality eval resources
Launch recorded by tools/run.py (artifacts/runs/20261010T104819Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-g --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run idle_vram_used_gib_max 23.6 GiB idle_cpu_cores_busy_mean 0.19 cores idle_cpu_cores_busy_p95 0.29 cores idle_gpu_util_mean_pct 0 % idle_pcie_rx_gbps_mean 0 GB/s idle_power_w_mean 109 W idle_ram_anon_gib 48.4 GiB "container cgroup memory.stat at phase end"idle_ram_file_gib 55.2 GiB "container cgroup memory.stat at phase end"prefill_vram_used_gib_max 23.6 GiB prefill_cpu_cores_busy_mean 1.29 cores prefill_cpu_cores_busy_p95 2.60 cores prefill_gpu_util_mean_pct 95.6 % prefill_pcie_rx_gbps_mean 11.6 GB/s prefill_power_w_mean 322 W prefill_ram_anon_gib 48.9 GiB "container cgroup memory.stat at phase end"prefill_ram_file_gib 55.2 GiB "container cgroup memory.stat at phase end"decode_vram_used_gib_max 23.7 GiB decode_cpu_cores_busy_mean 15.2 cores decode_cpu_cores_busy_p95 15.6 cores decode_gpu_util_mean_pct 99.6 % decode_pcie_rx_gbps_mean 3.47 GB/s decode_power_w_mean 359 W decode_ram_anon_gib 48.8 GiB "container cgroup memory.stat at phase end"decode_ram_file_gib 55.2 GiB "container cgroup memory.stat at phase end"prefill_tps_median 1,964 tok/s {"prompt_tokens": [26991, 27006, 26963]}decode_tps_median 93.8 tok/s {"completion_tokens": [1975, 1806, 1812]}
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 1ccc5e213b6654f6329de7c5db4e5183fe6b92c8 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T104851Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-gpu2-resource_profile.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261010T105104Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo ✓ pass load T44 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-10 20:51 AEST Finished 2026-10-10 20:53 AEST (2.2 min) Exit status None Notes –
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261010T105104Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 131 s host_ram_drop_gb 7.63 GB vram_ready_mib 22,374 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9035401e49a076325efdade467af0c5ddb3d62ca dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261010T105104Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-lo.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2
20261010T105317Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-re ✓ pass resources T44 eligibility: none
Config sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 Started 2026-10-10 20:53 AEST Finished 2026-10-10 20:55 AEST (2.6 min) Exit status 0 Notes resource profile; GPU index 2; probe 1569 samples
Other runs of this config: load load load load load lab ! lab quality eval needle ! needle ! resources
Launch recorded by tools/run.py (artifacts/runs/20261010T105104Z-lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-oma --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Digest sha256:84f75f3424c99a3d63392a4e0348292bdeba9cf7efba2ff1aa9b85c8fc0131e8Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k.json sha256 54e6390f3b25af9b0f38d2c701a92bb9baa6533ec691a2338c96390fdb47c91bMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 204800 --mem-fraction-static 0.80 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 5 --cuda-graph-max-bs-decode 1 --chunked-prefill-size 1024 --max-prefill-tokens 1024 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_ID
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run idle_vram_used_gib_max 22.3 GiB idle_cpu_cores_busy_mean 1.10 cores idle_cpu_cores_busy_p95 1.38 cores idle_gpu_util_mean_pct 0 % idle_pcie_rx_gbps_mean 0 GB/s idle_power_w_mean 113 W idle_ram_anon_gib 5.44 GiB "container cgroup memory.stat at phase end"idle_ram_file_gib 17.1 GiB "container cgroup memory.stat at phase end"prefill_vram_used_gib_max 22.6 GiB prefill_cpu_cores_busy_mean 1.10 cores prefill_cpu_cores_busy_p95 1.27 cores prefill_gpu_util_mean_pct 98.8 % prefill_pcie_rx_gbps_mean 0.18 GB/s prefill_power_w_mean 384 W prefill_ram_anon_gib 5.37 GiB "container cgroup memory.stat at phase end"prefill_ram_file_gib 17.1 GiB "container cgroup memory.stat at phase end"decode_vram_used_gib_max 22.6 GiB decode_cpu_cores_busy_mean 1.11 cores decode_cpu_cores_busy_p95 1.28 cores decode_gpu_util_mean_pct 95.3 % decode_pcie_rx_gbps_mean 2.06 GB/s decode_power_w_mean 388 W decode_ram_anon_gib 5.37 GiB "container cgroup memory.stat at phase end"decode_ram_file_gib 17.1 GiB "container cgroup memory.stat at phase end"prefill_tps_median 984 tok/s {"prompt_tokens": [26936, 27003, 26963]}decode_tps_median 108 tok/s {"completion_tokens": [2008, 1773, 1793]}mtp_acceptance_rate 0.4751 fraction {"mean_accept_len": 2.425, "batches": 57, "source": "SGLang decode log"}mtp_accepted_tps unavailable no decode speed in this step
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 9035401e49a076325efdade467af0c5ddb3d62ca dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261010T105317Z-sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2-re.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261010T105600Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass load T44 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-10 20:56 AEST Finished 2026-10-10 21:00 AEST (4.4 min) Exit status None Notes –
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261010T105600Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 264 s host_ram_drop_gb 64.2 GB vram_ready_mib 20,818 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit b37af885cdaa897b50aec9052c890862cfc967c5 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T105600Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2
20261010T110024Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe ✓ pass resources T44 eligibility: none
Config sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 Started 2026-10-10 21:00 AEST Finished 2026-10-10 21:03 AEST (3.2 min) Exit status 0 Notes resource profile; GPU index 2; probe 1900 samples
Other runs of this config: load load load load load load load lab lab lab lab lab lab lab lab kit sweep kit sweep kit sweep kit sweep TTFT TTFT TTFT TTFT stability ! stability ! stability ! stability ! quality eval quality panel quality panel quality panel needle ! needle resources
Launch recorded by tools/run.py (artifacts/runs/20261010T105600Z-lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30100 --shm-size 64g -e HF_HUB_OFFLINE=1 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e SGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 -e SGLANG_EXL3_MOE_OFFLOAD=gpu_cache -e EXL3_MOE_CPU_THREADS=24 -e SGLANG_EXL3_EXPERT_CACHE_GB=auto -e SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26 -e SGLANG_EXL3_KV_BITS=5 -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4 -e SGLANG_EXL3_OFFLOAD_FUSED=1 -e SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1 -e SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_NGRAM_TIER=nvme -e SGLANG_EXL3_VISION=1 -e SGLANG_EXL3_MM_FAST_CPU=1 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -v ~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coder Image ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Digest sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163Engine sglang Engine source https://github.com/0xSero/trellis-serve/tree/b1e74afcf98203dc5326349623b556ba2df4b16a Launch file launches/sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192.json sha256 8c27671e76ecf40389baef7e260b7f0cd955a9d90143e4d153acd115c0433587MTP / speculative off argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5 --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30100 --served-model-name flashnext --disable-shared-experts-fusion --kv-cache-dtype fp8_e4m3 --context-length 204800 --mem-fraction-static 0.88 --chunked-prefill-size 8192 --max-running-requests 4 --cuda-graph-max-bs-decode 8 --cuda-graph-backend-prefill disabled --max-mamba-cache-size 16 --max-total-tokens 210000 --reasoning-parser qwen3 --tool-call-parser qwen3_coderenv HF_HUB_OFFLINE=1CUDA_DEVICE_ORDER=PCI_BUS_IDSGLANG_EXL3_MODEL_PATH=/models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5SGLANG_EXL3_MOE_OFFLOAD=gpu_cacheEXL3_MOE_CPU_THREADS=24SGLANG_EXL3_EXPERT_CACHE_GB=autoSGLANG_EXL3_EXPERT_CACHE_RESERVE_GB=8.26SGLANG_EXL3_KV_BITS=5SGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_OFFLOAD_STAGING_PARTS=4SGLANG_EXL3_OFFLOAD_FUSED=1SGLANG_EXL3_OFFLOAD_COMPACT_PARTS=1SGLANG_EXL3_MOE_PREFILL_FP16_ACC=1SGLANG_EXL3_NGRAM_TIER=nvmeSGLANG_EXL3_VISION=1SGLANG_EXL3_MM_FAST_CPU=1SGLANG_EXL3_VIT_SDPA=1SGLANG_EXL3_VIT_MLP_CHUNK=8192
turboderp/Qwen3.8-Flash-Next-exl3 @69e33439ae950f17bcbe95c98f117d80f759ab6d · manifest models/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5.json
Metrics context.configured 204,800 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run idle_vram_used_gib_max 21.1 GiB idle_cpu_cores_busy_mean 1.20 cores idle_cpu_cores_busy_p95 2.10 cores idle_gpu_util_mean_pct 1.30 % idle_pcie_rx_gbps_mean 0.05 GB/s idle_power_w_mean 152 W idle_ram_anon_gib 57.2 GiB "container cgroup memory.stat at phase end"idle_ram_file_gib 54.8 GiB "container cgroup memory.stat at phase end"prefill_vram_used_gib_max 22.9 GiB prefill_cpu_cores_busy_mean 1.28 cores prefill_cpu_cores_busy_p95 1.81 cores prefill_gpu_util_mean_pct 74.5 % prefill_pcie_rx_gbps_mean 9.19 GB/s prefill_power_w_mean 279 W prefill_ram_anon_gib 57.5 GiB "container cgroup memory.stat at phase end"prefill_ram_file_gib 55.5 GiB "container cgroup memory.stat at phase end"decode_vram_used_gib_max 22.9 GiB decode_cpu_cores_busy_mean 1.26 cores decode_cpu_cores_busy_p95 1.61 cores decode_gpu_util_mean_pct 99.6 % decode_pcie_rx_gbps_mean 6.38 GB/s decode_power_w_mean 309 W decode_ram_anon_gib 57.4 GiB "container cgroup memory.stat at phase end"decode_ram_file_gib 55.8 GiB "container cgroup memory.stat at phase end"prefill_tps_median 2,131 tok/s {"prompt_tokens": [26967, 26935, 26998]}decode_tps_median 53.0 tok/s {"completion_tokens": [2039, 1749, 2048]}
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit b37af885cdaa897b50aec9052c890862cfc967c5 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261010T110024Z-sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efe.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2
20261011T004748Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-load ✓ pass load T45 eligibility: none
Config strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 10:47 AEST Finished 2026-10-11 10:48 AEST (0.5 min) Exit status None Notes –
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261011T004748Z-lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090- --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=coder -e MODEL=IQ1_M -e GGUF_DIR=/gguf/IQ1_M -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb.json sha256 21c445ce51f66e2e7e99721e02a761b0c7b3ef4c7d39c78392fc71a6361126b7MTP / speculative off argv env FAMILY=coderMODEL=IQ1_MGGUF_DIR=/gguf/IQ1_MCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF @5348543e0147355ac9cbcb031184a3546350988e · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 28.1 s host_ram_drop_gb 31.0 GB vram_ready_mib 23,662 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 13a7ca5bf6699db3c4f3d83874a810411069e2e4 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T004748Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2
20261011T004817Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-lab ✓ pass lab T45 eligibility: none
Config strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 10:48 AEST Finished 2026-10-11 10:50 AEST (2.4 min) Exit status 0 Notes served=qwen3.8-flash-next-coder-iq1_m
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261011T004748Z-lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090- --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=coder -e MODEL=IQ1_M -e GGUF_DIR=/gguf/IQ1_M -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb.json sha256 21c445ce51f66e2e7e99721e02a761b0c7b3ef4c7d39c78392fc71a6361126b7MTP / speculative off argv env FAMILY=coderMODEL=IQ1_MGGUF_DIR=/gguf/IQ1_MCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF @5348543e0147355ac9cbcb031184a3546350988e · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 93.2 tok/s lab_prefill_tps 2,657 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 13a7ca5bf6699db3c4f3d83874a810411069e2e4 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T004817Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2
20261011T005041Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-ttft ✓ pass TTFT T45 eligibility: none
Config strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 10:50 AEST Finished 2026-10-11 10:51 AEST (1.3 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab TTFT
Launch recorded by tools/run.py (artifacts/runs/20261011T004748Z-lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090- --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=coder -e MODEL=IQ1_M -e GGUF_DIR=/gguf/IQ1_M -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb.json sha256 21c445ce51f66e2e7e99721e02a761b0c7b3ef4c7d39c78392fc71a6361126b7MTP / speculative off argv env FAMILY=coderMODEL=IQ1_MGGUF_DIR=/gguf/IQ1_MCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF @5348543e0147355ac9cbcb031184a3546350988e · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,639 tok/s {"prompt_tokens": [13760, 13727, 13703], "ttft_s": [5.413, 5.202, 5.171]}ttft_prefill_32k_tps 2,761 tok/s {"prompt_tokens": [54631, 54664, 54748], "ttft_s": [19.784, 19.8, 19.857]}ttft_prefill_tps 2,639 tok/s {"prompt_tokens": [13760, 13727, 13703], "ttft_s": [5.413, 5.202, 5.171]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 13a7ca5bf6699db3c4f3d83874a810411069e2e4 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T005041Z-strata-qwen3.8-flash-next-coder-gsq-rco-iq1_m-128k-rtx-3090-24gb-gpu2-ttft.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T014733Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass load T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 11:47 AEST Finished 2026-10-11 11:48 AEST (0.6 min) Exit status None Notes –
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 34.1 s host_ram_drop_gb 59.9 GB vram_ready_mib 23,656 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit be97d5d0e733b936a850c94c188eefc4666db617 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T014733Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T014808Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass lab T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 11:48 AEST Finished 2026-10-11 11:55 AEST (7.0 min) Exit status 0 Notes served=orcarouter-qwen3.8-flash-next-uncensored-iq3_xxs
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 76.4 tok/s lab_prefill_tps 2,122 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit be97d5d0e733b936a850c94c188eefc4666db617 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T014808Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T015508Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass TTFT T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 11:55 AEST Finished 2026-10-11 11:56 AEST (1.6 min) Exit status 0 Notes headline length 8192; prompts carry random nonces; TTFT = first chunk with generated output (v2)
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run ttft_prefill_8k_tps 2,007 tok/s {"prompt_tokens": [13831, 13746, 13733], "ttft_s": [7.019, 6.827, 6.843]}ttft_prefill_32k_tps 2,162 tok/s {"prompt_tokens": [54799, 54714, 54691], "ttft_s": [25.269, 25.307, 25.318]}ttft_prefill_tps 2,007 tok/s {"prompt_tokens": [13831, 13746, 13733], "ttft_s": [7.019, 6.827, 6.843]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit be97d5d0e733b936a850c94c188eefc4666db617 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T015508Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T015645Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ! fail stability T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 11:56 AEST Finished 2026-10-11 12:06 AEST (10.1 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 50 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 2 repetition hit(s)
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 56,311 tokens context.headroom_min 73,648 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 50 calls {"per_min": 5.0, "requests": 35}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 2 hits {"ngram64x3": 2, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 86.0 tok/s window 1: 90.26 tok/s window 2: 94.88 tok/s window 3: 90.89 tok/s window 4: 96.59 tok/s window 5: 93.48 tok/s window 6: 90.89 tok/s window 7: 87.54 tok/s min 87.54 · max 96.59 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 383, "min_at_generation_s": 412.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 91.3 tok/s {"generation_seconds": 502.698, "generated_tokens": 45910.6}tasks_attempted 2 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 0 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": false, "resolved": false, "seconds": 41.8, "requests": 11, "tool_calls": 17, "completion_tokens": 1647, "occupied_max": 5130}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 558.2, "requests": 25, "tool_calls": 33, "completion_tokens": 44305, "occupied_max": 56311}] listtasks_over_64k [] instance idscompletion_tokens 45,952 tokens {"finish_reasons": {"tool_calls": 35}}mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Agentic summary: 600 s, 50 tool calls, 0 parse failures, 2 repetition hits, tasks solved 0/2, decode min window 86.0 tok/s, whole run 91.3 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit be97d5d0e733b936a850c94c188eefc4666db617 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T015645Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T020654Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass lab T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 12:06 AEST Finished 2026-10-11 12:11 AEST (4.6 min) Exit status 0 Notes served=orcarouter-qwen3.8-flash-next-uncensored-iq3_xxs
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 106,294 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 73.6 tok/s lab_prefill_tps 2,105 tok/s "context-gate prompt tokens / total request seconds"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit be97d5d0e733b936a850c94c188eefc4666db617 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T020654Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T021133Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass resources T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 12:11 AEST Finished 2026-10-11 12:13 AEST (2.0 min) Exit status 0 Notes resource profile; GPU index 2; probe 1218 samples
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run idle_vram_used_gib_max 23.7 GiB idle_cpu_cores_busy_mean 0.09 cores idle_cpu_cores_busy_p95 0.28 cores idle_gpu_util_mean_pct 0 % idle_pcie_rx_gbps_mean 0 GB/s idle_power_w_mean 120 W idle_ram_anon_gib 51.8 GiB "container cgroup memory.stat at phase end"idle_ram_file_gib 57.2 GiB "container cgroup memory.stat at phase end"prefill_vram_used_gib_max 23.7 GiB prefill_cpu_cores_busy_mean 1.34 cores prefill_cpu_cores_busy_p95 2.58 cores prefill_gpu_util_mean_pct 95.3 % prefill_pcie_rx_gbps_mean 12.5 GB/s prefill_power_w_mean 320 W prefill_ram_anon_gib 51.9 GiB "container cgroup memory.stat at phase end"prefill_ram_file_gib 57.2 GiB "container cgroup memory.stat at phase end"decode_vram_used_gib_max 23.7 GiB decode_cpu_cores_busy_mean 15.2 cores decode_cpu_cores_busy_p95 15.6 cores decode_gpu_util_mean_pct 99.9 % decode_pcie_rx_gbps_mean 3.81 GB/s decode_power_w_mean 352 W decode_ram_anon_gib 51.8 GiB "container cgroup memory.stat at phase end"decode_ram_file_gib 57.2 GiB "container cgroup memory.stat at phase end"prefill_tps_median 1,911 tok/s {"prompt_tokens": [26972, 26947, 26949]}decode_tps_median 92.8 tok/s {"completion_tokens": [1788, 1603, 1584]}
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit be97d5d0e733b936a850c94c188eefc4666db617 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T021133Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T021336Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass quality eval T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-11 12:13 AEST Finished 2026-10-12 00:41 AEST (748.0 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 127,837 tokens context.headroom_min 257 tokens tasks_attempted 50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 14 tasks {"eval_errors": 0}solve_rate 0.28 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 919.2, "requests": 57, "tool_calls": 79, "occupied_max": 127837}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": false, "seconds": 720.2, "requests": 24, "tool_calls": 28, "occupied_max": 42929}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 47816, "budget_exhausted": false, "seconds": 900.3, "requests": 65, "tool_calls": 91, "occupied_max": 92029}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 24615, "budget_exhausted": false, "seconds": 346.3, "requests": 22, "tool_calls": 34, "occupied_max": 40074}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18073, "budget_exhausted": false, "seconds": 948.4, "requests": 60, "tool_calls": 77, "occupied_max": 51047}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1069.6, "requests": 53, "tool_calls": 84, "occupied_max": 112191}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 816.9, "requests": 37, "tool_calls": 58, "occupied_max": 85619}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 880.4, "requests": 39, "tool_calls": 60, "occupied_max": 79018}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 53359, "budget_exhausted": false, "seconds": 765.0, "requests": 43, "tool_calls": 51, "occupied_max": 73796}, {"instance_id": "mesa__mesa-3892", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 775.9, "requests": 27, "tool_calls": 30, "occupied_max": 59330}, {"instance_id": "pytorch__rl-4210", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 757.1, "requests": 27, "tool_calls": 35, "occupied_max": 51443}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 23221, "budget_exhausted": false, "seconds": 281.7, "requests": 9, "tool_calls": 9, "occupied_max": 17707}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25561, "budget_exhausted": false, "seconds": 839.3, "requests": 39, "tool_calls": 54, "occupied_max": 48657}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 960.6, "requests": 39, "tool_calls": 55, "occupied_max": 68606}, {"instance_id": "domokane__FinancePy-317", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 929.3, "requests": 46, "tool_calls": 59, "occupied_max": 89011}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 52659, "budget_exhausted": false, "seconds": 691.1, "requests": 23, "tool_calls": 30, "occupied_max": 47569}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22123, "budget_exhausted": false, "seconds": 2276.9, "requests": 67, "tool_calls": 82, "occupied_max": 57552}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13455, "budget_exhausted": false, "seconds": 387.3, "requests": 28, "tool_calls": 42, "occupied_max": 33183}, {"instance_id": "docling-project__docling-4318", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 749.8, "requests": 34, "tool_calls": 33, "occupied_max": 55204}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 56528, "budget_exhausted": false, "seconds": 837.3, "requests": 47, "tool_calls": 70, "occupied_max": 77867}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Error:BadRequestError: litellm.BadRequestError: OpenAIException - prompt (126342 tokens) + max tokens (8192) exceeds the context (131072); requests are never truncated. Send a smaller max_tokens (at most 4722 here), or add \"fit_max_tokens\": true to the model's strata-<model>.json to shorten it to th", "submitted": false, "resolved": false, "completion_tokens": 51783, "budget_exhausted": false, "seconds": 1127.3, "requests": 60, "tool_calls": 106, "occupied_max": 121042}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 753.4, "requests": 22, "tool_calls": 38, "occupied_max": 73371}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 962.7, "requests": 46, "tool_calls": 55, "occupied_max": 78233}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 32200, "budget_exhausted": false, "seconds": 629.9, "requests": 44, "tool_calls": 71, "occupied_max": 73471}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1650.9, "requests": 74, "tool_calls": 113, "occupied_max": 113230}, {"instance_id": "marin-community__marin-9450", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 858.3, "requests": 37, "tool_calls": 71, "occupied_max": 101082}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 685.2, "requests": 22, "tool_calls": 28, "occupied_max": 54267}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 51535, "budget_exhausted": false, "seconds": 946.1, "requests": 72, "tool_calls": 102, "occupied_max": 100772}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 947.1, "requests": 94, "tool_calls": 112, "occupied_max": 106181}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4514, "budget_exhausted": false, "seconds": 156.0, "requests": 20, "tool_calls": 29, "occupied_max": 16235}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1441.7, "requests": 72, "tool_calls": 105, "occupied_max": 107895}, {"instance_id": "github__spec-kit-4559", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 890.8, "requests": 67, "tool_calls": 88, "occupied_max": 109247}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1134.9, "requests": 72, "tool_calls": 80, "occupied_max": 126826}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9105, "budget_exhausted": false, "seconds": 217.7, "requests": 34, "tool_calls": 57, "occupied_max": 29855}, {"instance_id": "dstackai__dstack-4256", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 59639, "budget_exhausted": false, "seconds": 827.2, "requests": 27, "tool_calls": 48, "occupied_max": 72666}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 967.9, "requests": 55, "tool_calls": 84, "occupied_max": 96984}, {"instance_id": "marin-community__marin-9419", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 872.3, "requests": 36, "tool_calls": 57, "occupied_max": 121388}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 20876, "budget_exhausted": false, "seconds": 1163.6, "requests": 40, "tool_calls": 53, "occupied_max": 47260}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 896.7, "requests": 31, "tool_calls": 52, "occupied_max": 72330}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 57557, "budget_exhausted": false, "seconds": 1692.2, "requests": 60, "tool_calls": 75, "occupied_max": 88901}, {"instance_id": "ansible__ansible-87624", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 844.8, "requests": 42, "tool_calls": 75, "occupied_max": 108710}, {"instance_id": "scipy__scipy-26156", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 795.2, "requests": 28, "tool_calls": 43, "occupied_max": 78576}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 985.5, "requests": 35, "tool_calls": 52, "occupied_max": 85773}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 776.0, "requests": 31, "tool_calls": 38, "occupied_max": 67419}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 813.7, "requests": 39, "tool_calls": 44, "occupied_max": 75902}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1399.1, "requests": 61, "tool_calls": 84, "occupied_max": 121670}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 971.2, "requests": 52, "tool_calls": 79, "occupied_max": 101253}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 17271, "budget_exhausted": false, "seconds": 392.0, "requests": 26, "tool_calls": 44, "occupied_max": 42651}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 1085.5, "requests": 62, "tool_calls": 99, "occupied_max": 111500}, {"instance_id": "roboflow__supervision-2600", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 774.7, "requests": 31, "tool_calls": 41, "occupied_max": 74528}] listtool_calls 3,084 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 14 responses {"wire": {"invalid_tool_call": 5, "unparsed_markup": 3}, "harness_format_errors": {"truncated_no_tool_call": 72, "no_tool_call": 2, "unknown_tool": 5, "missing_command": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 72 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 2 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 14 hits {"ngram64x3": 13, "same_call_x4": 1, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}decode_whole_run_tps 85.2 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 5a73ef5dd71c904cc9d6af503d7a53c401b53a98 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T021336Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T144136Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass needle T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-12 00:41 AEST Finished 2026-10-12 00:44 AEST (2.5 min) Exit status 0 Notes kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 104,552 tokens context.headroom_min unavailable no occupancy measured in this run needle_pass_rate 1.00 fraction [{"target_tokens": 98304, "prompt_tokens": 104407, "depth": 0.2, "code": "1050-8920", "answer": "1050-8920", "pass": true, "seconds": 49.5}, {"target_tokens": 98304, "prompt_tokens": 104487, "depth": 0.5, "code": "9578-9849", "answer": "9578-9849", "pass": true, "seconds": 49.0}, {"target_tokens": 98304, "prompt_tokens": 104552, "depth": 0.8, "code": "4287-2817", "answer": "4287-2817", "pass": true, "seconds": 49.2}]needle_max_passed_tokens 104,552 tokens
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 5a73ef5dd71c904cc9d6af503d7a53c401b53a98 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T144136Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2
20261011T144405Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2- ✓ pass quality eval T46 eligibility: none
Config strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 Started 2026-10-12 00:44 AEST Finished 2026-10-12 05:14 AEST (270.1 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load lab lab TTFT stability ! quality eval quality eval needle resources
Launch recorded by tools/run.py (artifacts/runs/20261011T014733Z-lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -w /opt/strata -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e PORT=8080 -v ~/models/local-ai-exp/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data --entrypoint /opt/strata/.venv/bin/python sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb.json sha256 7691cc5d3812511b6e4f7ec33b5375a697b84063ee93000127aae22f9ebbbefcMTP / speculative off argv /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /data/config/strata-orca-iq3_xxs.json --port 8080 --gpu 0env CUDA_DEVICE_ORDER=PCI_BUS_IDPORT=8080
orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF @e43d00f4e2b8b40b89f75e9adeb1045ac34c8acc · manifest models/orcarouter-Qwen3.8-Flash-Next-Uncensored-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max 94,117 tokens context.headroom_min 36,842 tokens tasks_attempted 30 tasks {"task_ids": ["matplotlib__matplotlib-26011", "sympy__sympy-20442", "psf__requests-2148", "scikit-learn__scikit-learn-25570", "django__django-14752", "django__django-13158", "sympy__sympy-21171", "sympy__sympy-18199", "django__django-11049", "django__django-13028", "django__django-13551", "pallets__flask-4045", "sympy__sympy-19487", "mwaskom__seaborn-3190", "sympy__sympy-12171", "sphinx-doc__sphinx-7738", "scikit-learn__scikit-learn-13497", "django__django-16873", "astropy__astropy-6938", "django__django-14667", "scikit-learn__scikit-learn-11040", "sphinx-doc__sphinx-10325", "scikit-learn__scikit-learn-11281", "psf__requests-1963", "scikit-learn__scikit-learn-13439", "matplotlib__matplotlib-25498", "django__django-14672", "django__django-11964", "django__django-16408", "django__django-14787"], "task_set_sha256": "faf83c53cbf1ee8067d029b07e9874583a01af5fb4dd2346278389a8298f8cee", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 21 tasks {"eval_errors": 0}solve_rate 0.7 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "matplotlib__matplotlib-26011", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17294, "budget_exhausted": false, "seconds": 508.7, "requests": 29, "tool_calls": 38, "occupied_max": 30585}, {"instance_id": "sympy__sympy-20442", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 882.2, "requests": 43, "tool_calls": 60, "occupied_max": 84206}, {"instance_id": "psf__requests-2148", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29915, "budget_exhausted": false, "seconds": 514.5, "requests": 35, "tool_calls": 58, "occupied_max": 49083}, {"instance_id": "scikit-learn__scikit-learn-25570", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": false, "seconds": 758.7, "requests": 21, "tool_calls": 23, "occupied_max": 57688}, {"instance_id": "django__django-14752", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6529, "budget_exhausted": false, "seconds": 130.3, "requests": 19, "tool_calls": 24, "occupied_max": 13813}, {"instance_id": "django__django-13158", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 31740, "budget_exhausted": false, "seconds": 550.3, "requests": 67, "tool_calls": 90, "occupied_max": 72248}, {"instance_id": "sympy__sympy-21171", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25802, "budget_exhausted": false, "seconds": 396.0, "requests": 17, "tool_calls": 29, "occupied_max": 34952}, {"instance_id": "sympy__sympy-18199", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 742.1, "requests": 26, "tool_calls": 37, "occupied_max": 60261}, {"instance_id": "django__django-11049", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15357, "budget_exhausted": false, "seconds": 257.9, "requests": 24, "tool_calls": 40, "occupied_max": 30358}, {"instance_id": "django__django-13028", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 11509, "budget_exhausted": false, "seconds": 193.2, "requests": 22, "tool_calls": 29, "occupied_max": 26770}, {"instance_id": "django__django-13551", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17337, "budget_exhausted": false, "seconds": 289.6, "requests": 25, "tool_calls": 33, "occupied_max": 34039}, {"instance_id": "pallets__flask-4045", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14445, "budget_exhausted": false, "seconds": 280.3, "requests": 37, "tool_calls": 41, "occupied_max": 34051}, {"instance_id": "sympy__sympy-19487", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10445, "budget_exhausted": false, "seconds": 176.5, "requests": 29, "tool_calls": 29, "occupied_max": 21988}, {"instance_id": "mwaskom__seaborn-3190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 50151, "budget_exhausted": false, "seconds": 806.7, "requests": 66, "tool_calls": 103, "occupied_max": 85662}, {"instance_id": "sympy__sympy-12171", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 772.1, "requests": 30, "tool_calls": 45, "occupied_max": 80377}, {"instance_id": "sphinx-doc__sphinx-7738", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 825.8, "requests": 61, "tool_calls": 75, "occupied_max": 76415}, {"instance_id": "scikit-learn__scikit-learn-13497", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 27782, "budget_exhausted": false, "seconds": 443.7, "requests": 51, "tool_calls": 55, "occupied_max": 51571}, {"instance_id": "django__django-16873", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 751.6, "requests": 29, "tool_calls": 40, "occupied_max": 63077}, {"instance_id": "astropy__astropy-6938", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 906.0, "requests": 39, "tool_calls": 52, "occupied_max": 76037}, {"instance_id": "django__django-14667", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 32715, "budget_exhausted": false, "seconds": 516.7, "requests": 51, "tool_calls": 60, "occupied_max": 57005}, {"instance_id": "scikit-learn__scikit-learn-11040", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12439, "budget_exhausted": false, "seconds": 220.2, "requests": 19, "tool_calls": 34, "occupied_max": 28899}, {"instance_id": "sphinx-doc__sphinx-10325", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28600, "budget_exhausted": false, "seconds": 501.1, "requests": 56, "tool_calls": 78, "occupied_max": 59565}, {"instance_id": "scikit-learn__scikit-learn-11281", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 795.2, "requests": 38, "tool_calls": 50, "occupied_max": 76143}, {"instance_id": "psf__requests-1963", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 40374, "budget_exhausted": false, "seconds": 703.0, "requests": 38, "tool_calls": 56, "occupied_max": 59447}, {"instance_id": "scikit-learn__scikit-learn-13439", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4831, "budget_exhausted": false, "seconds": 91.5, "requests": 15, "tool_calls": 21, "occupied_max": 10864}, {"instance_id": "matplotlib__matplotlib-25498", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 35357, "budget_exhausted": false, "seconds": 558.6, "requests": 37, "tool_calls": 51, "occupied_max": 64685}, {"instance_id": "django__django-14672", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 802.5, "requests": 33, "tool_calls": 33, "occupied_max": 72991}, {"instance_id": "django__django-11964", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22364, "budget_exhausted": false, "seconds": 283.2, "requests": 14, "tool_calls": 20, "occupied_max": 19070}, {"instance_id": "django__django-16408", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 56596, "budget_exhausted": false, "seconds": 865.1, "requests": 60, "tool_calls": 78, "occupied_max": 94117}, {"instance_id": "django__django-14787", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17585, "budget_exhausted": false, "seconds": 264.5, "requests": 24, "tool_calls": 36, "occupied_max": 31663}] listtool_calls 1,418 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 5 responses {"wire": {"invalid_tool_call": 4}, "harness_format_errors": {"truncated_no_tool_call": 18, "invalid_arguments": 1, "unknown_tool": 1}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 18 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 21 hits {"ngram64x3": 21, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "tiktoken cl100k_base (approximation: server exposes no tokenize endpoint)"}decode_whole_run_tps 86.0 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 5a73ef5dd71c904cc9d6af503d7a53c401b53a98 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T144405Z-strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2-.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb
20261011T191447Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-load ✓ pass load T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb Started 2026-10-12 05:14 AEST Finished 2026-10-12 05:15 AEST (0.6 min) Exit status None Notes –
Other runs of this config: load lab
Launch recorded by tools/run.py (artifacts/runs/20261011T191447Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 34.1 s host_ram_drop_gb 55.6 GB vram_ready_mib 23,660 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 3d9a6cab88549c415b7c1800e61a577ac052a277 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T191447Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb
20261011T191522Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-lab ✓ pass lab T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb Started 2026-10-12 05:15 AEST Finished 2026-10-12 05:17 AEST (2.1 min) Exit status 0 Notes official lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-iq3-s-128k-rtx-3090-24gb.128k.20261011T191729.json; recipe written only on pass
Other runs of this config: load lab
Launch recorded by tools/run.py (artifacts/runs/20261011T191447Z-lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=IQ3_S -e GGUF_DIR=/gguf/IQ3_S -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb.json sha256 ecc9982b8e42ed6d57c36fe2a4570b7b6b7e54cf51ea3dd1a194a489c182c7fbMTP / speculative off argv env FAMILY=qwenMODEL=IQ3_SGGUF_DIR=/gguf/IQ3_SCONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 75.0 tok/s lab_prefill_tps 2,174 tok/s "official lab.py try"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 3d9a6cab88549c415b7c1800e61a577ac052a277 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T191522Z-strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
20261011T191739Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load ✓ pass load T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb Started 2026-10-12 05:17 AEST Finished 2026-10-12 05:18 AEST (0.4 min) Exit status None Notes –
Other runs of this config: load load lab lab lab lab
Launch recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 25.0 s host_ram_drop_gb 39.6 GB vram_ready_mib 23,646 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 778868340d6979684600554d81a6f1354d7a3a8d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T191739Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
20261011T191806Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab ✓ pass lab T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb Started 2026-10-12 05:18 AEST Finished 2026-10-12 05:20 AEST (2.7 min) Exit status 0 Notes official lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261011T192046.json; recipe written only on pass
Other runs of this config: load load lab lab lab lab
Launch recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 103 tok/s lab_prefill_tps 2,644 tok/s "official lab.py try"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 778868340d6979684600554d81a6f1354d7a3a8d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T191806Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
20261011T192047Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab ✓ pass lab T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb Started 2026-10-12 05:20 AEST Finished 2026-10-12 05:23 AEST (3.0 min) Exit status 0 Notes official lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261011T192346.json; recipe written only on pass
Other runs of this config: load load lab lab lab lab
Launch recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 101 tok/s lab_prefill_tps 2,631 tok/s "official lab.py try"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 778868340d6979684600554d81a6f1354d7a3a8d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T192047Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
20261011T192347Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab ✓ pass lab T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb Started 2026-10-12 05:23 AEST Finished 2026-10-12 05:26 AEST (3.0 min) Exit status 0 Notes official lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261011T192647.json; recipe written only on pass
Other runs of this config: load load lab lab lab lab
Launch recorded by tools/run.py (artifacts/runs/20261011T191739Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 88.2 tok/s lab_prefill_tps 2,638 tok/s "official lab.py try"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 778868340d6979684600554d81a6f1354d7a3a8d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261011T192347Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
20261011T192654Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-load ✓ pass load T48 eligibility: none
Config sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 Started 2026-10-12 05:26 AEST Finished 2026-10-12 05:28 AEST (1.9 min) Exit status None Notes –
Other runs of this config: load lab lab stability quality eval
Launch recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Digest sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cfMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv CUDA_DEVICE_ORDER=PCI_BUS_IDHF_HUB_OFFLINE=1PYTORCH_CUDA_ALLOC_CONF=expandable_segments:TrueSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_PREFILL_FP16_ACC=1SGLANG_EXL3_VIT_MLP_CHUNK=8192SGLANG_EXL3_VIT_SDPA=1SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 212,992 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 113 s host_ram_drop_gb 11.1 GB vram_ready_mib 21,270 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit e265bcccc6f30794949bee93596736937e910e5d dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261011T192654Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-load.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
20261011T192849Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab ✓ pass lab T48 eligibility: none
Config sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 Started 2026-10-12 05:28 AEST Finished 2026-10-12 05:33 AEST (5.1 min) Exit status 0 Notes served=turboderp-Qwen3.8-27B-exl3-3.00bpw
Other runs of this config: load lab lab stability quality eval
Launch recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Digest sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cfMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv CUDA_DEVICE_ORDER=PCI_BUS_IDHF_HUB_OFFLINE=1PYTORCH_CUDA_ALLOC_CONF=expandable_segments:TrueSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_PREFILL_FP16_ACC=1SGLANG_EXL3_VIT_MLP_CHUNK=8192SGLANG_EXL3_VIT_SDPA=1SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 212,992 tokens context.occupied_max 173,372 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 126 tok/s lab_prefill_tps 831 tok/s "context-gate prompt tokens / total request seconds"mtp_acceptance_rate 0.7354 fraction {"mean_accept_len": 3.207, "batches": 101, "source": "SGLang decode log"}mtp_accepted_tps 126 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261011T192849Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
20261011T193357Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-stability ✓ pass stability T48 eligibility: none
Config sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 Started 2026-10-12 05:33 AEST Finished 2026-10-12 05:44 AEST (10.6 min) Exit status 0 Notes stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 134 tool calls in this run count toward the config's 100-call minimum (§5)
Other runs of this config: load lab lab stability quality eval
Launch recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Digest sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cfMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv CUDA_DEVICE_ORDER=PCI_BUS_IDHF_HUB_OFFLINE=1PYTORCH_CUDA_ALLOC_CONF=expandable_segments:TrueSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_PREFILL_FP16_ACC=1SGLANG_EXL3_VIT_MLP_CHUNK=8192SGLANG_EXL3_VIT_SDPA=1SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 212,992 tokens context.occupied_max 68,013 tokens context.headroom_min 144,269 tokens duration_s 600 s {"cap_s": 600.0, "driver_capped": true}tool_calls 134 calls {"per_min": 13.4, "requests": 107}parse_failures 0 responses {"wire": {}, "harness_format_errors": {}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 0 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 0 hits {"ngram64x3": 0, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize", "saved": "artifacts: proxy/repetition/"}decode_window_min_tps 90.1 tok/s window 1: 112.88 tok/s window 2: 116.11 tok/s window 3: 112.69 tok/s window 4: 102.4 tok/s window 5: 99.84 tok/s window 6: 116.3 tok/s min 99.84 · max 116.3 tok/s {"window_s": 60, "skip_s": 60, "rolling_step_s": 1, "n_rolling_windows": 333, "min_at_generation_s": 329.0, "windows_tps_note": "consecutive non-overlapping 60 s windows; the value is the minimum over rolling windows (1 s step)"}decode_whole_run_tps 113 tok/s {"generation_seconds": 452.722, "generated_tokens": 51092.2}tasks_attempted 3 attempts {"unfinished_at_cap": 1, "task_ids": ["django__django-11099", "sympy__sympy-21612", "psf__requests-2317", "matplotlib__matplotlib-23299", "scikit-learn__scikit-learn-13439", "astropy__astropy-12907", "pytest-dev__pytest-7373", "sympy__sympy-20590"], "task_set_sha256": "8e7487ca7d059df6359fc7d9b86c7021256a8a81b8af049616750f8f83400324"}tasks_solved 2 attempts {"eval_errors": 0, "swebench": "5.0.2"}crash_or_oom no {"upstream_connection_failures": 0, "upstream_http_errors": 0, "source": "proxy-observed; the run wrapper adds container death/OOM"}gates_after skipped filled by qualify step: the lab run that follows this run (tools/qualify.py) agent_attempts [{"attempt": 1, "round": 1, "instance_id": "django__django-11099", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 52.3, "requests": 16, "tool_calls": 20, "completion_tokens": 3350, "occupied_max": 11234}, {"attempt": 2, "round": 1, "instance_id": "sympy__sympy-21612", "exit_status": "Submitted", "submitted": true, "resolved": true, "seconds": 457.9, "requests": 55, "tool_calls": 66, "completion_tokens": 40203, "occupied_max": 68013}, {"attempt": 3, "round": 1, "instance_id": "psf__requests-2317", "exit_status": "WallClockCap", "submitted": false, "resolved": false, "seconds": 89.8, "requests": 37, "tool_calls": 49, "completion_tokens": 8143, "occupied_max": 20955}] listtasks_over_64k ["sympy__sympy-21612"] instance idscompletion_tokens 51,696 tokens {"finish_reasons": {"tool_calls": 107}}mtp_acceptance_rate 0.6951 fraction {"mean_accept_len": 3.086, "batches": 421, "source": "SGLang decode log"}mtp_accepted_tps 113 tok/s "output tok/s including accepted draft tokens"
Agentic summary: 600 s, 134 tool calls, 0 parse failures, 0 repetition hits, tasks solved 2/3, decode min window 90.1 tok/s, whole run 113 tok/s.
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261011T193357Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-stability.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
20261011T194435Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab ✓ pass lab T48 eligibility: none
Config sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 Started 2026-10-12 05:44 AEST Finished 2026-10-12 05:49 AEST (4.6 min) Exit status 0 Notes served=turboderp-Qwen3.8-27B-exl3-3.00bpw
Other runs of this config: load lab lab stability quality eval
Launch recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Digest sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cfMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv CUDA_DEVICE_ORDER=PCI_BUS_IDHF_HUB_OFFLINE=1PYTORCH_CUDA_ALLOC_CONF=expandable_segments:TrueSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_PREFILL_FP16_ACC=1SGLANG_EXL3_VIT_MLP_CHUNK=8192SGLANG_EXL3_VIT_SDPA=1SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 212,992 tokens context.occupied_max 173,372 tokens context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 139 tok/s lab_prefill_tps 836 tok/s "context-gate prompt tokens / total request seconds"mtp_acceptance_rate 0.8036 fraction {"mean_accept_len": 3.413, "batches": 75, "source": "SGLang decode log"}mtp_accepted_tps 139 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261011T194435Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-lab.json
Runs / Qwen3.8-27B / sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2
20261011T194914Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-quality_eval ✓ pass quality eval T48 eligibility: none
Config sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 Started 2026-10-12 05:49 AEST Finished 2026-10-12 11:47 AEST (358.4 min) Exit status 0 Notes quality evaluation, uncapped, 60000 completion tokens per task; mini-swe-agent 04d809ceab9d; sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | tool calls in quality_eval runs do not count toward the §5 100-call minimum
Other runs of this config: load lab lab stability quality eval
Launch recorded by tools/run.py (artifacts/runs/20261011T192654Z-lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2/docker-run.txt) Copy
docker run -d --name lab-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2 --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:30000 --shm-size 16g -e CUDA_DEVICE_ORDER=PCI_BUS_ID -e HF_HUB_OFFLINE=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_EXL3_EMBED_HOST=1 -e SGLANG_EXL3_PREFILL_FP16_ACC=1 -e SGLANG_EXL3_VIT_MLP_CHUNK=8192 -e SGLANG_EXL3_VIT_SDPA=1 -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1 -e SGLANG_VLM_CACHE_SIZE_MB=256 -v ~/models/local-ai-exp/turboderp-Qwen3.8-27B-exl3-3.00bpw-src:/models:ro --entrypoint /opt/entrypoint.sh ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79 python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.pt Image ghcr.io/0xsero/sglang-exl3@sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Digest sha256:c8922bd7256caf1b273af7ac49221e652be9e6072c1cdec4b959f506fd415d79Engine sglang-exl3 Engine source – Launch file launches/sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb.json sha256 11948b11dab392d837fae6ef37fb0fcd96c03ec96372fc16afa89ce2505a35cfMTP / speculative {"algorithm": "NEXTN", "num_steps": 3, "eagle_topk": 1, "draft_tokens": 4, "token_map": "qwen38_hot32k_v2.pt"}argv /opt/entrypoint.sh python3 -m sglang.launch_server --model-path /models/turboderp-Qwen3.8-27B-exl3-3.00bpw --quantization exl3 --trust-remote-code --host 0.0.0.0 --port 30000 --served-model-name turboderp-Qwen3.8-27B-exl3-3.00bpw --context-length 212992 --mem-fraction-static 0.82 --kv-cache-dtype fp8_e4m3 --mamba-ssm-dtype bfloat16 --max-running-requests 1 --max-mamba-cache-size 3 --cuda-graph-max-bs-decode 1 --cuda-graph-max-bs-prefill 240 --chunked-prefill-size 4096 --max-prefill-tokens 4096 --prefill-decode-interval 16 --reasoning-parser qwen3 --tool-call-parser qwen3_coder --mamba-radix-cache-strategy no_buffer --disable-overlap-schedule --image-processor-backend pil --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --speculative-token-map /opt/sglang-exl3/tokenmaps/qwen38_hot32k_v2.ptenv CUDA_DEVICE_ORDER=PCI_BUS_IDHF_HUB_OFFLINE=1PYTORCH_CUDA_ALLOC_CONF=expandable_segments:TrueSGLANG_EXL3_EMBED_HOST=1SGLANG_EXL3_PREFILL_FP16_ACC=1SGLANG_EXL3_VIT_MLP_CHUNK=8192SGLANG_EXL3_VIT_SDPA=1SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=1SGLANG_VLM_CACHE_SIZE_MB=256
turboderp/Qwen3.8-27B-exl3 @6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 · manifest models/turboderp-Qwen3.8-27B-exl3-3.00bpw-src.json
Metrics context.configured 212,992 tokens context.occupied_max 99,327 tokens context.headroom_min 109,482 tokens tasks_attempted 50 tasks {"task_ids": ["bramstroker__homeassistant-powercalc-4795", "pandas-dev__pandas-68003", "UKGovernmentBEIS__inspect_ai-5264", "apache__airflow-72774", "UKGovernmentBEIS__inspect_ai-5190", "zauberzeug__nicegui-6380", "WeblateOrg__weblate-21643", "nilearn__nilearn-6608", "marimo-team__marimo-10788", "mesa__mesa-3892", "pytorch__rl-4210", "more-itertools__more-itertools-1248", "datalayer__jupyter-mcp-server-427", "kyegomez__swarms-2374", "domokane__FinancePy-317", "arc53__DocsGPT-2772", "optiland__optiland-796", "locustio__locust-3523", "docling-project__docling-4318", "pallets__werkzeug-3298", "pipeshub-ai__pipeshub-ai-3244", "holoviz__holoviews-7058", "supabase__supabase-py-1619", "agno-agi__agno-9887", "openai__openai-agents-python-5016", "marin-community__marin-9450", "google-deepmind__mujoco_warp-1660", "LearningCircuit__local-deep-research-6226", "ansible-collections__community.general-12804", "embeddings-benchmark__mteb-5443", "nautobot__nautobot-9531", "github__spec-kit-4559", "Genesis-Embodied-AI__genesis-world-3399", "fossasia__eventyay-5507", "dstackai__dstack-4256", "ogx-ai__ogx-6725", "marin-community__marin-9419", "pymc-labs__pymc-marketing-3099", "fsspec__filesystem_spec-2131", "stickerdaniel__linkedin-mcp-server-973", "ansible__ansible-87624", "scipy__scipy-26156", "roboflow__rf-detr-1502", "scrapy__scrapy-8138", "WeblateOrg__weblate-21746", "wkentaro__labelme-2667", "gdsfactory__gdsfactory-4840", "cognizant-ai-lab__neuro-san-studio-1452", "OpenHands__software-agent-sdk-4871", "roboflow__supervision-2600"], "task_set_sha256": "b28ea453590fb21d8d9375c95519cae00599595b3d5ca85a011d872db16b560b", "sampling_sha256": "fdcbc89037b763c925766cce107e85ae1097c3456888ffa25108f2380b940b22", "swebench": "5.0.2"}tasks_solved 39 tasks {"eval_errors": 1}solve_rate 0.78 fraction token_budget_per_task 60,000 completion tokens per_task [{"instance_id": "bramstroker__homeassistant-powercalc-4795", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 29667, "budget_exhausted": false, "seconds": 508.4, "requests": 61, "tool_calls": 83, "occupied_max": 68863}, {"instance_id": "pandas-dev__pandas-68003", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 12864, "budget_exhausted": false, "seconds": 260.1, "requests": 28, "tool_calls": 37, "occupied_max": 32478}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5264", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22223, "budget_exhausted": false, "seconds": 507.7, "requests": 76, "tool_calls": 83, "occupied_max": 54810}, {"instance_id": "apache__airflow-72774", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 7718, "budget_exhausted": false, "seconds": 74.2, "requests": 16, "tool_calls": 23, "occupied_max": 17777}, {"instance_id": "UKGovernmentBEIS__inspect_ai-5190", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9397, "budget_exhausted": false, "seconds": 193.1, "requests": 32, "tool_calls": 40, "occupied_max": 28500}, {"instance_id": "zauberzeug__nicegui-6380", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 49962, "budget_exhausted": false, "seconds": 641.9, "requests": 56, "tool_calls": 61, "occupied_max": 71289}, {"instance_id": "WeblateOrg__weblate-21643", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 17446, "budget_exhausted": false, "seconds": 438.2, "requests": 49, "tool_calls": 56, "occupied_max": 36321}, {"instance_id": "nilearn__nilearn-6608", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 32018, "budget_exhausted": false, "seconds": 358.4, "requests": 39, "tool_calls": 49, "occupied_max": 49496}, {"instance_id": "marimo-team__marimo-10788", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19250, "budget_exhausted": false, "seconds": 220.9, "requests": 34, "tool_calls": 48, "occupied_max": 43629}, {"instance_id": "mesa__mesa-3892", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 15718, "budget_exhausted": false, "seconds": 177.1, "requests": 26, "tool_calls": 34, "occupied_max": 39389}, {"instance_id": "pytorch__rl-4210", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12790, "budget_exhausted": false, "seconds": 169.4, "requests": 21, "tool_calls": 22, "occupied_max": 22905}, {"instance_id": "more-itertools__more-itertools-1248", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 44893, "budget_exhausted": false, "seconds": 553.5, "requests": 53, "tool_calls": 60, "occupied_max": 68495}, {"instance_id": "datalayer__jupyter-mcp-server-427", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16488, "budget_exhausted": false, "seconds": 214.2, "requests": 22, "tool_calls": 32, "occupied_max": 30899}, {"instance_id": "kyegomez__swarms-2374", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13996, "budget_exhausted": false, "seconds": 226.0, "requests": 36, "tool_calls": 43, "occupied_max": 29618}, {"instance_id": "domokane__FinancePy-317", "exit_status": "Submitted", "submitted": true, "resolved": null, "completion_tokens": 12596, "budget_exhausted": false, "seconds": 169.1, "requests": 28, "tool_calls": 33, "occupied_max": 31333}, {"instance_id": "arc53__DocsGPT-2772", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 18143, "budget_exhausted": false, "seconds": 258.2, "requests": 17, "tool_calls": 21, "occupied_max": 19196}, {"instance_id": "optiland__optiland-796", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 14547, "budget_exhausted": false, "seconds": 331.5, "requests": 36, "tool_calls": 48, "occupied_max": 36517}, {"instance_id": "locustio__locust-3523", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 6212, "budget_exhausted": false, "seconds": 105.4, "requests": 19, "tool_calls": 25, "occupied_max": 19479}, {"instance_id": "docling-project__docling-4318", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 42365, "budget_exhausted": false, "seconds": 461.9, "requests": 39, "tool_calls": 53, "occupied_max": 62092}, {"instance_id": "pallets__werkzeug-3298", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 12636, "budget_exhausted": false, "seconds": 125.4, "requests": 18, "tool_calls": 28, "occupied_max": 25241}, {"instance_id": "pipeshub-ai__pipeshub-ai-3244", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 55326, "budget_exhausted": false, "seconds": 849.2, "requests": 62, "tool_calls": 68, "occupied_max": 90422}, {"instance_id": "holoviz__holoviews-7058", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 37313, "budget_exhausted": false, "seconds": 425.6, "requests": 42, "tool_calls": 56, "occupied_max": 39298}, {"instance_id": "supabase__supabase-py-1619", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25453, "budget_exhausted": false, "seconds": 230.8, "requests": 36, "tool_calls": 47, "occupied_max": 31915}, {"instance_id": "agno-agi__agno-9887", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 33850, "budget_exhausted": false, "seconds": 343.0, "requests": 34, "tool_calls": 43, "occupied_max": 54816}, {"instance_id": "openai__openai-agents-python-5016", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 10138, "budget_exhausted": false, "seconds": 149.1, "requests": 29, "tool_calls": 37, "occupied_max": 24960}, {"instance_id": "marin-community__marin-9450", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 22085, "budget_exhausted": false, "seconds": 312.5, "requests": 39, "tool_calls": 59, "occupied_max": 50513}, {"instance_id": "google-deepmind__mujoco_warp-1660", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 641.4, "requests": 34, "tool_calls": 42, "occupied_max": 54175}, {"instance_id": "LearningCircuit__local-deep-research-6226", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 38476, "budget_exhausted": false, "seconds": 452.1, "requests": 50, "tool_calls": 72, "occupied_max": 64416}, {"instance_id": "ansible-collections__community.general-12804", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 52535, "budget_exhausted": false, "seconds": 612.0, "requests": 87, "tool_calls": 109, "occupied_max": 83204}, {"instance_id": "embeddings-benchmark__mteb-5443", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 4988, "budget_exhausted": false, "seconds": 190.4, "requests": 21, "tool_calls": 27, "occupied_max": 16013}, {"instance_id": "nautobot__nautobot-9531", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 23809, "budget_exhausted": false, "seconds": 1257.9, "requests": 109, "tool_calls": 127, "occupied_max": 62642}, {"instance_id": "github__spec-kit-4559", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 28165, "budget_exhausted": false, "seconds": 540.4, "requests": 60, "tool_calls": 83, "occupied_max": 67025}, {"instance_id": "Genesis-Embodied-AI__genesis-world-3399", "exit_status": "RepeatedFormatError", "submitted": false, "resolved": false, "completion_tokens": 57625, "budget_exhausted": false, "seconds": 644.5, "requests": 42, "tool_calls": 38, "occupied_max": 54696}, {"instance_id": "fossasia__eventyay-5507", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8598, "budget_exhausted": false, "seconds": 194.9, "requests": 40, "tool_calls": 50, "occupied_max": 25888}, {"instance_id": "dstackai__dstack-4256", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 13442, "budget_exhausted": false, "seconds": 197.9, "requests": 41, "tool_calls": 53, "occupied_max": 36955}, {"instance_id": "ogx-ai__ogx-6725", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 25221, "budget_exhausted": false, "seconds": 322.6, "requests": 42, "tool_calls": 53, "occupied_max": 55458}, {"instance_id": "marin-community__marin-9419", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 45787, "budget_exhausted": false, "seconds": 510.6, "requests": 36, "tool_calls": 44, "occupied_max": 72796}, {"instance_id": "pymc-labs__pymc-marketing-3099", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 19860, "budget_exhausted": false, "seconds": 1117.5, "requests": 74, "tool_calls": 85, "occupied_max": 55676}, {"instance_id": "fsspec__filesystem_spec-2131", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 8397, "budget_exhausted": false, "seconds": 153.6, "requests": 15, "tool_calls": 21, "occupied_max": 17233}, {"instance_id": "stickerdaniel__linkedin-mcp-server-973", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16622, "budget_exhausted": false, "seconds": 230.5, "requests": 24, "tool_calls": 30, "occupied_max": 39013}, {"instance_id": "ansible__ansible-87624", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 41882, "budget_exhausted": false, "seconds": 491.0, "requests": 83, "tool_calls": 114, "occupied_max": 77462}, {"instance_id": "scipy__scipy-26156", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 26034, "budget_exhausted": false, "seconds": 465.9, "requests": 47, "tool_calls": 60, "occupied_max": 41930}, {"instance_id": "roboflow__rf-detr-1502", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 9951, "budget_exhausted": false, "seconds": 252.9, "requests": 32, "tool_calls": 50, "occupied_max": 26069}, {"instance_id": "scrapy__scrapy-8138", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 16914, "budget_exhausted": false, "seconds": 164.0, "requests": 22, "tool_calls": 27, "occupied_max": 28474}, {"instance_id": "WeblateOrg__weblate-21746", "exit_status": "Submitted", "submitted": true, "resolved": true, "completion_tokens": 34673, "budget_exhausted": false, "seconds": 790.4, "requests": 61, "tool_calls": 68, "occupied_max": 63418}, {"instance_id": "wkentaro__labelme-2667", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 805.0, "requests": 54, "tool_calls": 60, "occupied_max": 63073}, {"instance_id": "gdsfactory__gdsfactory-4840", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 36398, "budget_exhausted": false, "seconds": 464.3, "requests": 49, "tool_calls": 65, "occupied_max": 59467}, {"instance_id": "cognizant-ai-lab__neuro-san-studio-1452", "exit_status": "Submitted", "submitted": true, "resolved": false, "completion_tokens": 11499, "budget_exhausted": false, "seconds": 135.0, "requests": 21, "tool_calls": 31, "occupied_max": 27207}, {"instance_id": "OpenHands__software-agent-sdk-4871", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 811.1, "requests": 61, "tool_calls": 82, "occupied_max": 99327}, {"instance_id": "roboflow__supervision-2600", "exit_status": "TokenBudgetExhausted", "submitted": false, "resolved": false, "completion_tokens": 60000, "budget_exhausted": true, "seconds": 538.5, "requests": 20, "tool_calls": 18, "occupied_max": 33907}] listtool_calls 2,598 calls {"counted_toward_100_minimum": false, "why": "tool calls in quality_eval runs do not count toward the \u00a75 100-call minimum"}parse_failures 1 responses {"wire": {"invalid_tool_call": 1}, "harness_format_errors": {"truncated_no_tool_call": 27}, "union_rule": "a response counts once even if both the wire check and the harness flag it", "counted": "malformed tool calls only (decision 0003); truncations and other format errors are separate metrics"}output_truncations 27 responses {"rule": "cut off at max_tokens with no tool call"}agent_format_errors 0 responses {"rule": "agent protocol errors other than malformed tool calls"}repetition_hits 43 hits {"ngram64x3": 43, "same_call_x4": 0, "ngram64x3_cross_field_not_counted": 0, "tokenizer": "server /v1/tokenize"}decode_whole_run_tps 112 tok/s kld unavailable not produced by the agentic evaluation; see quality_panel / KLD records top1_agreement unavailable not produced by the agentic evaluation; see quality_panel records mtp_acceptance_rate 0.6948 fraction {"mean_accept_len": 3.085, "batches": 10995, "source": "SGLang decode log"}mtp_accepted_tps 112 tok/s "output tok/s including accepted draft tokens"
Cache, layout and host Cache warm Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 7c4216a051bd1d711db4765e3f24c8e0b48da6d7 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-27b/20261011T194914Z-sglang-qwen3.8-27b-registry-208k-rtx-3090-24gb-gpu2-quality_eval.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
20261012T014808Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load ✓ pass load T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb Started 2026-10-12 11:48 AEST Finished 2026-10-12 11:48 AEST (0.4 min) Exit status None Notes –
Other runs of this config: load load lab lab lab lab
Launch recorded by tools/run.py (artifacts/runs/20261012T014808Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run load_seconds 25.0 s host_ram_drop_gb 39.4 GB vram_ready_mib 23,646 MiB
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 47c9862242d864e21d266cc77304505cf845d531 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261012T014808Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-load.json
Runs / Qwen3.8-Flash-Next / strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb
20261012T014835Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab ✓ pass lab T47 eligibility: none
Config strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb Started 2026-10-12 11:48 AEST Finished 2026-10-12 11:50 AEST (2.0 min) Exit status 0 Notes official lab.py try --on endpoint; run file rtx-3090-24gb.qwen3.8-flash-next.strata-qwen3.8-flash-next-q2-0-128k-rtx-3090-24gb.128k.20261012T015035.json; recipe written only on pass
Other runs of this config: load load lab lab lab lab
Launch recorded by tools/run.py (artifacts/runs/20261012T014808Z-lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb/docker-run.txt) Copy
docker run -d --name lab-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb --security-opt no-new-privileges --gpus device=GPU-76d3c6af-7f18-69d2-3145-899103de1722 -p 127.0.0.1:18080:8080 --shm-size 16g --ulimit memlock=-1 --stop-timeout 60 -e FAMILY=qwen -e MODEL=Q2_0 -e GGUF_DIR=/gguf/Q2_0 -e CONTEXT=131072 -e VISION=no -e KV=int8 -e GPU=0 -e LOW_RAM=off -e PORT=8080 -e CUDA_DEVICE_ORDER=PCI_BUS_ID -v ~/models/local-ai-exp/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF:/gguf:ro -v ~/models/local-ai-exp/strata-data:/data sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172 Image sha256:dcc2e2e0893d790f843251b5031a8070f5bc4db2389218fbe52f4b484e9ce172Digest –Engine strata Engine source https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c Launch file launches/strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb.json sha256 aadd2d65b0c6365622705da0d70416e9f7ede7673d2676c9a02bb8ae37403e0fMTP / speculative off argv env FAMILY=qwenMODEL=Q2_0GGUF_DIR=/gguf/Q2_0CONTEXT=131072VISION=noKV=int8GPU=0LOW_RAM=offPORT=8080CUDA_DEVICE_ORDER=PCI_BUS_ID
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ed59f92082b1e93c0e96d60a8b11aab089b52f09 · manifest models/ISTA-DASLab-Qwen3.8-Flash-Next-GSQ-RCO-GGUF.json
Metrics context.configured 131,072 tokens context.occupied_max unavailable no occupancy measured in this run context.headroom_min unavailable no occupancy measured in this run gates_passed ["load", "chat", "reasoning", "tools", "context", "speed"] gates {"load": true, "chat": true, "reasoning": true, "tools": true, "context": true, "speed": true}lab_decode_c1_tps 99.7 tok/s lab_prefill_tps 2,664 tok/s "official lab.py try"mtp_accepted_tps skipped speculative decoding off in this launch mtp_acceptance_rate skipped speculative decoding off in this launch
Cache, layout and host Cache cold · reset: container, prefix_kv, expert_cache, ngram_row_cache, page_cache Layout current · 1 card(s) used · display on hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 cpus_offline 2,18 cpu_boost False cpu_max_mhz 3401 machine_checks_this_boot 6
# Used Name UUID Bus PCIe Display 0 NVIDIA GeForce RTX 3090 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off 1 NVIDIA GeForce RTX 3090 GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on 2 ● NVIDIA GeForce RTX 3090 GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
Provenance bench commit 47c9862242d864e21d266cc77304505cf845d531 dirty tree pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78 (matches current pins.json)registry d21258dd744e7c78be28177c90060c6af8e10b7ekit ef883d269e50ecbea290f919f095e2f3ca633b42mini-swe-agent 04d809ceab9df28f9adaed044884180159172930
Artifacts and raw output Raw record: results/qwen3.8-flash-next/20261012T014835Z-strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-lab.json
Superseded. This page is kept for old links; the coding results below are memorised; see fresh-issues results .
Qwen3.8-Flash-Next vs Qwen3.8-27B Exploratory deployment comparison on this box (decision 0014): each model as best deployed on one RTX 3090 at PCIe 4.0 x8 (GPU2), same harness, task set, sampling and token budget. Flash-Next: PR #144 launch on a21efea6 with the stability fix (chunked prefill 8192). 27B: the operator's daily server (EXL3 3.00 bpw, NEXTN/MTP, single stream). Conclusions are per priority, not "better overall".
Coding quality on this page is memorised. The SWE-bench-Lite tasks below are public; Flash-Next reproduces the original fixes almost word for word. The contamination-reduced comparison is on the Fresh-issues benchmark page.
Metric Flash-Next 27B Agentic coding — 30 SWE-bench-Lite tasks, one uncapped attempt each Tasks solved 30 18 Solve rate 100.0% 60.0% Tool calls 1,445 1,453 Parse failures 1 5 Repetition hits 16 72 Decode during the run 42.3 tok/s 125.5 tok/s 10-minute agentic run (8-task §5 set) Tool calls 69 137 Slowest 60-s window 39.8 tok/s 111.1 tok/s Whole-run decode 43.6 tok/s 119.7 tok/s Repetition hits 0 1 Speed (kit protocol, early exit off; lab gates) Decode, 1 request 47.5 tok/s 120.9 tok/s Decode, 1 request at 32k 48.5 tok/s 111.6 tok/s Decode, 4 requests (aggregate) 54.3 tok/s 122.2 tok/s Prefill 8k 2,285 tok/s 1,076 tok/s Prefill 64k 2,431 tok/s 854 tok/s TTFT prefill 8k 2,284 tok/s 1,034 tok/s Lab decode C1 48.8 tok/s 143.7 tok/s Lab prefill 2,131 tok/s 740 tok/s MTP acceptance — 56.3% Context and footprint Real KV capacity (server log) 209,984 tokens 135,424 tokens Needle pass rate 100.0% 75.0% Largest needle passed 191,045 tokens 104,552 tokens Load time 260 s 127 s Host RAM used 65.6 GB 7.5 GB VRAM at ready 20,844 MiB 22,038 MiB
Contamination check SWE-bench-Lite fixes are public since 2023, so a model may have seen them in training. For each task we measured the share of the real upstream fix's added lines that the model's patch reproduces verbatim. A model that solves by recall reproduces the gold patch; one that solves by reasoning writes its own.
Model Median share of the gold fix reproduced Tasks with ≥ 90 % reproduced Flash-Next 100 % 24 of 30 27B 14 % 4 of 27
Reading: a high share means the solve rate largely measures memorisation of these public fixes, not agentic problem-solving. The solve rates below are reported as measured, with this caveat; a decontaminated task set (issues newer than both models) is the fair test.
Paired outcome on the same 30 tasks Flash-Next only: 12 · 27B only: 0 · both or neither: 18. Solve-rate difference (Flash-Next − 27B) +40.0 pp (95 % CI +22.5 to +57.5, Wald); exact McNemar p = 0.000488 .
Task Flash-Next tokens min 27B tokens min matplotlib__matplotlib-26011 ✓ solved · 86 % gold 24,487 21.4 ✓ solved · 14 % gold 21,395 5.0 sympy__sympy-20442 ✓ solved · 20 % gold 14,321 10.8 ✓ solved · 20 % gold 19,992 3.5 psf__requests-2148 ✓ solved · 100 % gold 11,569 8.7 ✓ solved · 100 % gold 54,056 8.9 scikit-learn__scikit-learn-25570 ✓ solved · 0 % gold 20,399 12.9 ✓ solved · 0 % gold 33,030 5.8 django__django-14752 ✓ solved · 22 % gold 9,370 5.6 ✓ solved · 33 % gold 9,909 1.7 django__django-13158 ✓ solved · 100 % gold 21,298 14.9 ✓ solved · 0 % gold 27,576 4.7 sympy__sympy-21171 ✓ solved · 100 % gold 9,622 6.2 ✗ · 33 % gold 4,996 1.9 sympy__sympy-18199 ✓ solved · 96 % gold 38,145 23.1 ✗ · 2 % gold 16,378 2.9 django__django-11049 ✓ solved · 100 % gold 9,853 5.4 ✓ solved · 100 % gold 10,825 1.8 django__django-13028 ✓ solved · 100 % gold 5,844 3.6 ✓ solved · 0 % gold 29,907 5.5 django__django-13551 ✓ solved · 100 % gold 12,939 7.9 ✗ · 0 % gold 28,779 4.2 pallets__flask-4045 ✓ solved · 100 % gold 10,158 6.3 ✗ · 17 % gold 14,338 2.8 sympy__sympy-19487 ✓ solved · 100 % gold 9,242 24.2 ✗ · 50 % gold 11,456 2.2 mwaskom__seaborn-3190 ✓ solved · 100 % gold 12,532 14.6 ✗ · 0 % gold 50,913 11.0 sympy__sympy-12171 ✓ solved · 100 % gold 18,740 9.5 ✗ 60,000 8.4 sphinx-doc__sphinx-7738 ✓ solved · 100 % gold 11,528 6.6 ✓ solved · 100 % gold 48,319 9.1 scikit-learn__scikit-learn-13497 ✓ solved · 100 % gold 15,258 7.3 ✓ solved · 12 % gold 7,993 2.0 django__django-16873 ✓ solved · 33 % gold 11,438 6.0 ✓ solved · 33 % gold 11,280 1.9 astropy__astropy-6938 ✓ solved · 100 % gold 27,193 15.2 ✓ solved · 0 % gold 14,524 3.9 django__django-14667 ✓ solved · 100 % gold 11,426 8.1 ✗ · 0 % gold 10,764 2.0 scikit-learn__scikit-learn-11040 ✓ solved · 100 % gold 10,873 5.8 ✗ · 18 % gold 19,349 3.5 sphinx-doc__sphinx-10325 ✓ solved · 100 % gold 18,642 13.4 ✓ solved · 14 % gold 36,683 6.3 scikit-learn__scikit-learn-11281 ✓ solved · 96 % gold 22,233 12.9 ✓ solved · 48 % gold 25,193 4.0 psf__requests-1963 ✓ solved · 100 % gold 11,156 9.2 ✓ solved · 0 % gold 34,857 6.4 scikit-learn__scikit-learn-13439 ✓ solved · 40 % gold 9,148 5.4 ✓ solved · 40 % gold 2,815 0.5 matplotlib__matplotlib-25498 ✓ solved · 100 % gold 26,215 18.9 ✗ 60,000 8.0 django__django-14672 ✓ solved · 100 % gold 6,092 3.5 ✓ solved · 100 % gold 6,628 1.3 django__django-11964 ✓ solved · 100 % gold 15,049 7.6 ✗ · 0 % gold 54,124 8.6 django__django-16408 ✓ solved · 100 % gold 20,168 12.3 ✗ 60,000 9.6 django__django-14787 ✓ solved · 100 % gold 10,487 6.5 ✓ solved · 0 % gold 24,950 4.2
Hardware & methodology Host-to-device bandwidth Run Outcome Layout GPU Bus PCIe H2D D2H Notes 2026-10-03 06:53 AEST ✓ pass current GPU2 GPU-76d3c6af-7f18-69d2-3145-899103de1722 00000000:0C:00.0gen4 x8 13.5 GB/s 13.2 GB/s 1 GiB pinned transfers; full table in detail 2026-10-03 06:53 AEST ✓ pass current GPU0 GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d 00000000:04:00.0gen4 x8 6.03 GB/s 6.60 GB/s 1 GiB pinned transfers; full table in detail 2026-10-04 19:11 AEST ✓ pass current GPU1 GPU-c67ac872-3371-a88c-aa92-d969daf6405e 00000000:0B:00.0gen4 x8 13.5 GB/s 13.2 GB/s 1 GiB pinned transfers; full table in detail
NVMe random reads Run Outcome IOPS Throughput Detail Notes 2026-10-03 06:54 AEST ✓ pass 168,600 IOPS 3.35 GB/s "16 KiB, 32 threads"~/models/local-ai-exp/turboderp-Qwen3.8-Flash-Next-exl3-3.05bpw_h5_ng5/ngram_embedding.safetensors
Host hostname omarchy-gpu cpu AMD Ryzen 9 5950X 16-Core Processor ram_total_gib 125.7 ram_layout DIMM_A1 32 GiB DDR4 3200 MT/s, DIMM_A2 32 GiB DDR4 3200 MT/s, DIMM_B1 32 GiB DDR4 3200 MT/s, DIMM_B2 32 GiB DDR4 3200 MT/s (AM4: 2 channels) kernel 7.2.5-3-omarchy driver 610.57.04 cuda 13.3 cpus_online 0-1,3-17,19-31 0-31 (changed between runs) cpus_offline 2,18 none (changed between runs) machine_checks_this_boot 0 6 (changed between runs) cpu_boost False cpu_max_mhz 3401
Layout GPU UUID Bus PCIe Display current GPU-816b43e4-8d65-dfd7-2234-8517b3dfbf2d00000000:04:00.0gen4 x8 off current GPU-c67ac872-3371-a88c-aa92-d969daf6405e00000000:0B:00.0gen4 x8 on current GPU-76d3c6af-7f18-69d2-3145-899103de172200000000:0C:00.0gen4 x8 off
GPUs as seen by the records (PCIe width is the link width at record time).
Dropped configs and why Model Config Failed runs Why (record notes) GLM exllamav3-glm-5.3-flash-exl3-3.05bpw-exact-128k-1x-gpu2 ! fail lab / gates ! fail TTFT / invalid measurement ! fail stability ! fail lab / gatesserved=glm-5.3-flash INVALIDATED: TTFT measured to an empty role chunk the exllamav3 server sends before prefill (ttft.py v1 bug); re-measured with v2. headline length 8192; prompts carry random nonces stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 21 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 3.356 < floor 15.0 served=glm-5.3-flash GLM glm-llamacpp-iq2m-128k-1x-gpu2 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.91 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM glm53-offload-v4.1-nvme-55g-rtx-3090-24gb-gpu2 ! fail lab / gates ! fail kit sweep / timeoutserved=glm-5.3-flash (no notes) GLM glm53-offload-v4.1-nvme-90g-rtx-3090-24gb-gpu2 ! fail lab / gatesserved=glm-5.3-flash GLM ik_llama-glm-5.3-flash-iq2-m-128k-1x-gpu2 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 47 tool-call parse failure(s); decode window min 9.458 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-ckpt4-gpu2 ! fail lab / gates ✕ crash stability / measureserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation) GLM ik_llama-glm-5.3-flash-iq2-m-128k-1x-mtp-gpu2 ! fail lab / gates ✕ crash stability / measureserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation) GLM ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-fm4096-gpu012 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 68 tool-call parse failure(s); decode window min 13.255 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM ik_llama-glm-5.3-flash-iq2-m-128k-3x-g210-gpu012 ✕ crash load / loadche_init: CUDA2 KV buffer size = 570.81 MiB
llama_init_from_model: KV self size = 748.00 MiB, c^KV (q8_0): 748.00 MiB, kv^T: not used
llama_init_from_model: KV self indexer size = 704.00 MiB (f16)
llama_init_from_model: CUDA_Host output buffer size = 0.59 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696
llama_init_from_model: failed to allocate compute buffers
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'
GLM ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm4096-gpu012 ✕ crash load / load c^KV (q8_0): 68.00 MiB, kv^T: not used
llama_init_from_model: KV self indexer size = 64.00 MiB (f16)
llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2754.25 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 2888039424
llama_init_from_model: failed to allocate compute buffers
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
common_speculative_init: failed to create MTP context
srv init: failed to initialize recurrent speculative context
~ggml_backend_cuda_context: have 13 graphs
~ggml_backend_cuda_context: have 14 graphs
~ggml_backend_cuda_context: have 11 graphs
GLM ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-fm7168-gpu012 ! fail lab / gates ✕ crash stability / measureserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); 9 tool-call parse failure(s); decode floor not verifiable (too little generation) GLM ik_llama-glm-5.3-flash-iq2-m-128k-3x-mtp-g210-gpu012 ✕ crash load / loadche_init: CUDA2 KV buffer size = 570.81 MiB
llama_init_from_model: KV self size = 816.00 MiB, c^KV (q8_0): 816.00 MiB, kv^T: not used
llama_init_from_model: KV self indexer size = 768.00 MiB (f16)
llama_init_from_model: CUDA_Host output buffer size = 0.61 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 3488.51 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 3657965696
llama_init_from_model: failed to allocate compute buffers
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
~ggml_backend_cuda_context: have 0 graphs
llama_init_from_gpt_params: error: failed to create context with model '/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf'
GLM llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-gpu2 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 30 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.854 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub2048-t15-gpu2 ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub4096-gpu2 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 27 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 8.847 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq2-m-128k-1x-ub8192-gpu2 ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-g210-gpu012 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 24 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.035 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-fitt2048-gpu012 ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub2048-gpu012 ! fail lab / gates ! fail stability ✕ crash lab / harnessserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 26 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 11.388 < floor 15.0 (no notes) GLM llama.cpp-glm-5.3-flash-iq2-m-128k-3x-ub4096-fitt2048-g210-gpu012 ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ2_M/GLM-5.3-Flash-BF16-IQ2_M-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq3-xxs-128k-1x-gpu2 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.646 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq3-xxs-128k-3x-ub2048-gpu012 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 22 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 9.855 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ3_XXS/GLM-5.3-Flash-BF16-IQ3_XXS-00001-of-00004.gguf GLM llama.cpp-glm-5.3-flash-iq4-xs-128k-3x-ub2048-fitt2048-g210-gpu012 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 25 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 7.807 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-IQ4_XS/GLM-5.3-Flash-BF16-IQ4_XS-00001-of-00005.gguf GLM llama.cpp-glm-5.3-flash-q2-k-128k-1x-gpu2 ! fail lab / gates ! fail stability ! fail lab / gatesserved=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf stability run, 10 min cap, floor 15.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 19 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 8.8 < floor 15.0 served=/models/GLM-5.3-Flash-BF16-Q2_K/GLM-5.3-Flash-BF16-Q2_K-00001-of-00004.gguf 27B sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-ctx131k-gpu2 ! fail lab / gates ! fail needle / measure ! fail needle / measure(no notes) kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint 27B sglang-qwen3.8-27b-exl3-3.00bpw-nextn-200k-rtx-3090-24gb-omarchy-gpu2 ! fail lab / gates ! fail needle / measure ! fail stability ! fail lab / gates(no notes) kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 137 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s) (no notes) Flash-Next fn-llamacpp-iq4xs-128k-1x-fit-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 49 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 1 repetition hit(s); decode window min 23.83 < floor 50.0 Flash-Next fn-llamacpp-iq4xs-128k-1x-gpu2 ✕ crash load / loaduser, abort
0.18.994.031 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 65358.17 MiB on device 0: cudaMalloc failed: out of memory
0.18.994.037 E alloc_tensor_range: failed to allocate CUDA0 buffer of size 68533006336
0.19.139.695 E llama_model_load: error loading model: unable to allocate CUDA0 buffer
0.19.139.702 E llama_model_load_from_file_impl: failed to load model
0.19.139.708 E cmn common_init_: failed to load model '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf'
0.19.139.713 E srv load_model: failed to load model, '/models/Qwen3.8-Flash-Next-IQ4_XS/Qwen3.8-Flash-Next-IQ4_XS-00001-of-00003.gguf'
0.19.139.715 I srv operator(): operator(): cleaning up before exit...
0.19.140.570 E srv llama_server: exiting due to model loading error
Flash-Next fn-trellis-3.05-0xsero-main-x8-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 63 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.077 < floor 50.0 Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-fitt2048-g210-gpu012 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 70 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.626 < floor 50.0 Flash-Next llama.cpp-qwen3.8-flash-next-iq4xs-128k-3x-fit-gpu012 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode floor not verifiable (too little generation); agent driver exited 1 Flash-Next sglang-qwen3.8-flash-next-exl3-2.05bpw-offload-200k-rtx-3090-24gb-gpu2 ✕ crash load / load_glue/offload_moe_method.py", line 86, in get_runtime
store = HostExpertStore(model_dir, layers, num_experts, hidden, inter, bits, prefix=prefix)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/trellis-serve/cuda/src/sglang_exl3/kernels/offload_store.py", line 62, in __init__
raise NotImplementedError("HostExpertStore: fast relayout implemented for K=3 experts only")
NotImplementedError: HostExpertStore: fast relayout implemented for K=3 experts only
[2026-10-04 10:55:46] Received sigquit from a child process. It usually means the child failed.
[2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=True, pid=1
[2026-10-04 10:55:46] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-gpu2 ! fail stability ! fail stability ! fail stability ! fail needle / measure ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 64 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.539 < floor 50.0 stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 62 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.784 < floor 50.0 stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 61 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.37 < floor 50.0 kit needle method (filler + one fact, thinking off, T=0), served model name from the endpoint stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 69 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.78 < floor 50.0 Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-c8192-probe-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 53 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 38.954 < floor 50.0 Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-g1-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 57 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.08 < floor 50.0 Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-gpu2 ✕ crash stability / measure ! fail stability ✕ crash stability / measurestability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 0 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation) stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 39.676 < floor 50.0 stability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 6 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: engine connection failures (crash/OOM); decode floor not verifiable (too little generation) Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-pinned-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 72 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 40.915 < floor 50.0 Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r6-gpu2 ✕ crash lab / harness ✕ crash TTFT / measure(no notes) (no notes) Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-a21efea6-r926-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 71 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.429 < floor 50.0 Flash-Next sglang-qwen3.8-flash-next-exl3-3.05bpw-offload-200k-rtx-3090-24gb-phase-d-f77c72f6-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: server /v1/tokenize | 67 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: decode window min 37.939 < floor 50.0 Flash-Next sglang-qwen3.8-flash-next-exl3-4.05bpw-offload-200k-rtx-3090-24gb-gpu2 ✕ crash load / load ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 116, in __init__
t = parse_table(model_path)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/trellis-serve/cuda/src/sglang_exl3/offload/ngram_nvme.py", line 94, in parse_table
raise ValueError(f"{path}: shard_N.trellis tensors missing or not contiguous")
ValueError: /models/turboderp-Qwen3.8-Flash-Next-exl3-4.05bpw_h6_ng6/ngram_embedding.safetensors: shard_N.trellis tensors missing or not contiguous
[2026-10-04 10:56:54] Received sigquit from a child process. It usually means the child failed.
[2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=True, pid=1
[2026-10-04 10:56:54] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Flash-Next strata-qwen3.8-flash-next-gsq-rco-q2_0-128k-rtx-3090-24gb-gpu2 ! fail kit sweep / measuresweep status: None Flash-Next strata-qwen3.8-flash-next-uncensored-orcarouter-iq3_xxs-128k-rtx-3090-24gb-gpu2 ! fail stabilitystability run, 10 min cap, floor 50.0 tok/s; mini-swe-agent 04d809ceab9d (benchmarks/swebench.yaml, native tool calls); sampling {"temperature": 0.7, "top_p": 0.95, "seed": 1234, "max_tokens": 8192} | repetition tokenizer: tiktoken cl100k_base (approximation: server exposes no tokenize endpoint) | 50 tool calls in this run count toward the config's 100-call minimum (§5) | FAIL: 2 repetition hit(s)
All failed and crashed runs (84) Decisions 0001-flashnext-llamacpp-quant.md 0001 — Flash-Next llama.cpp comparison quant (T11)0001 — Flash-Next llama.cpp comparison quant (T11)
Decision: use bartowski/Qwen3.8-Flash-Next-GGUF@928589fdb66c6ff07f22ac561e3fbce76553548f IQ4_XS (97.7 GB) for both the 1-card and the 3-card llama.cpp comparison, on the attested ghcr.io/ggml-org/llama.cpp:server-cuda12-b11312 image. N-gram table (per_layer_token_embd.weight) pinned to CPU with -ot; experts on CPU for the layers that do not fit (--n-cpu-moe N, tuned once per layout), fp16→q8_0 KV only if needed for 128k+.
Why: closest to 4-bit that fits 24 GB VRAM + ~110 GiB RAM on one card (IQ4_XS ≈ 91 GiB); same file serves the 3-card run, so the 1 vs 3 comparison changes only the layout. Q4_K_M (119.6 GB ≈ 111 GiB) would not leave room on 1 card. No MTP (llama.cpp Flash-Next MTP PR #27836 not merged). Not REAP-pruned.
Limits: the kit quality panel speaks the SGLang API only, so this route gets no kit band → it cannot be the Flash-Next headline (SPEC §5); it is a comparison point. Delete after T11 unless it beats trellis.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
IQ4_XS stays the paired 1 vs 3 card file. Q4_K_M only as a single feasibility screen if IQ4_XS is competitive with trellis (disk is the binding constraint).
The 1 vs 3 pair freezes occupied context, KV format, batching and workload; --n-cpu-moe / tensor split per layout are reported, and any KV change is a separate comparison.
Full digest + shard hashes via tools/models.py; tensor placement of per_layer_token_embd.weight verified from the server log.
Eligibility experiment-only (no kit band). Results kept regardless; weights deleted only after manifests exist.
0001-flashnext-llamacpp-quant.review.md 0001-flashnext-llamacpp-quant.review
AMEND. Keep IQ4_XS as the primary paired comparison, but replace the categorical Q4_K_M rejection with a memory-budget check. The strongest counter-argument is that the decision compares the entire file against host RAM , despite offloading weights to VRAM.
Q4_K_M is not demonstrably excluded. 119.6 GB ≈ 111.4 GiB . If 18–20 GiB of weights reside on the GPU and their host pages are reclaimable, host weight residency becomes roughly 91–93 GiB , leaving 17–19 GiB within the stated 110 GiB RAM budget. Loading peaks, pinned allocations, KV and scratch could consume that margin; the supplied numbers do not prove they will.
Concrete alternative: use IQ4_XS for T11, with one bounded Q4_K_M feasibility screen at the intended context if the allocation budget supports it. Keep any Q4 results as a separate quant comparison. Do not expand into another tuning grid.
Fitting is not evidence of adequate speed. Dual-channel DDR4-3200 has a theoretical 51.2 GB/s ceiling. At 50 tok/s, that permits only 1.024 GB of DRAM traffic per token . If all 6B active LM parameters required fresh ideal 4-bit reads, that is ~3 GB/token and at most 17 tok/s before overhead. Record CPU offload placement and measured decode; this is a conditional bandwidth screen, not a performance prediction.
“Only the layout changes” needs qualification. Independently tuning --n-cpu-moe N compares each layout’s selected configuration, rather than isolating card-count scaling. Freeze occupied context, KV format, batching and workload across the pair; report both CPU-expert allocations and tensor splits. Treat any KV-format switch as a separate comparison.
Complete provenance before launch. Pin the full image digest already supplied in SPEC §6, not just server-cuda12-b11312. Add the exact GGUF shard list and hashes, immutable launches, and the actual tensor-placement evidence for per_layer_token_embd.weight. The proposed -ot setting remains unverified in this review.
The quality limitation is correctly identified, but applies to registry eligibility too. Without the required kit band, this Flash-Next configuration is experiment-only , even if all registry lab gates pass. Neither IQ4_XS nor a speed win establishes quality equivalence.
Replace “delete unless it beats trellis.” Define the comparison at matched occupied context and cache state, and retain results regardless of outcome. Delete rejected weights only after hashes and immutable references are recorded; retain any finalist under §9. A useful three-card result need not beat a single-GPU engine to justify publication.
0002-glm-quant-plan.md 0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)0002 — GLM-5.3-Flash quant plan for llama.cpp / ik_llama.cpp (T20–T22)
Source: bartowski/GLM-5.3-Flash-BF16-GGUF@66e5e9f0e337470e8801b99bae6c45779b042cc2 (made with llama.cpp b11279, mainline-compatible; MTP layers kept at Q4_0). Unsloth GGUFs excluded: not loadable on mainline until re-uploaded.
Layout Quants Size Fit reasoning 1 card (24 GB VRAM + ~115 GiB usable RAM) IQ2_M, Q2_K 120.6 / 125.7 GB (112 / 117 GiB) ~21 GiB on GPU, rest in RAM 1 card, stretch IQ3_XXS 138.8 GB (129 GiB) over RAM by ~10 GiB → mmap paging from NVMe; run once, expect slow, record it 3 cards (72 GB VRAM + RAM) IQ3_XXS, IQ4_XS 138.8 / 177.7 GB IQ4_XS ≈ 165 GiB: ~66 GiB VRAM + ~99 GiB RAM; ~4-bit like every accepted GLM recipe
Order (disk is ~750 GB for everything, so sequential): IQ2_M → Q2_K → (IQ3_XXS) → delete the losing 2-bit → IQ4_XS. ik_llama.cpp + MTP runs on the same files.
Quality judged per SPEC §5 (tools gate, 0 parse failures over ≥100 calls, ≥80 % of reference solve rate in the uncapped eval).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Order: IQ2_M and Q2_K on 1 card → same files on 3 cards → IQ3_XXS on 1 card (tight resident candidate, not presumed paging) and 3 cards → IQ4_XS on 3 cards.
Fit is checked by measured placement at the stated context, per GPU; uneven splits watched.
Nothing is a finalist before the uncapped quality evaluation; deletion only of rejected candidates, after manifests.
MTP on/off is its own variable family; MTP use verified from engine logs and metrics.
Publish tool-call counts and reference solve counts next to every quality verdict.
0002-glm-quant-plan.review.md 0002-glm-quant-plan.review
AMEND. Keep the candidate quants, but correct the IQ3_XXS fit claim and make deletion depend on qualification. The strongest counter-argument is that weight capacity does not establish useful speed or quality : the spec already estimates one-card GLM at 8–13 tok/s, below the 15 tok/s registry floor.
IQ3_XXS is not automatically a paging experiment. Using the decision’s assumptions: 138.8 GB ≈ 129.3 GiB ; subtracting 21 GiB GPU residency leaves 108.3 GiB in RAM , about 6.7 GiB below the stated 115 GiB usable budget. Treat it as a tight resident-memory candidate. Declare paging only after accounting for context, runtime allocations and actual tensor placement.
All fit claims need a specified context and measured placement. Approximate host-weight budgets are 91.3 GiB for IQ2_M , 96.1 GiB for Q2_K , and 99.5 GiB for three-card IQ4_XS . Their remaining host margins are roughly 23.7, 18.9 and 15.5 GiB respectively. These exclude additional runtime memory. Likewise, allocating 66 GiB across three cards leaves only 2 GiB per card on average ; an uneven split can exhaust one GPU.
Budget bandwidth before promising 15 tok/s. Dual-channel DDR4-3200 has a theoretical maximum of 51.2 GB/s . At 15 tok/s, that permits at most 3.41 GB of DRAM traffic per generated token , with a smaller practical allowance. Measure effective bandwidth, expert placement and decode; VRAM capacity alone cannot show that the one-card route clears this constraint. Also, Q2_K is only 4.2% larger than IQ2_M—kernel efficiency and quality could easily determine the winner.
“~4-bit like accepted recipes” is precedent, not qualification. The spec’s reported sub-4-bit KLD of 0.28–0.38 makes quality a substantial risk. Establish the reference explicitly: exact EXL3 if loadable, otherwise the highest-bit GGUF that fits. If that is IQ4_XS, retain it through all candidate evaluations. A speed winner cannot be called a finalist before the uncapped quality evaluation.
Concrete alternative order: screen IQ2_M and Q2_K on one card, compare those files on three cards, then screen IQ3_XXS on both layouts and IQ4_XS on three cards. Run uncapped quality evaluation for promising configurations, then qualification for survivors. Give one-card IQ3_XXS an ordinary residency check before labelling it “stretch”; make three-card IQ3_XXS an explicit comparison point.
Replace “delete the losing 2-bit” with “delete a rejected candidate.” All four listed quants total 562.8 GB , leaving about 185 GB of the specified 748 GB free before Qwen weights, images and other artifacts. Build the combined inventory first. Preserve hashes and file references before deletion, and retain qualifying finalists under §9; being slower than another quant does not itself establish rejection.
Shared files do not establish shared MTP support. Verify that each pinned engine loads this exact revision and actually uses the retained MTP layer. Compare MTP off/on as a separate variable family; record accepted tokens/s, acceptance rate, settings and quality. Keep locally built ik_llama.cpp results experiment-only until its image provenance becomes eligible.
The decision’s quality summary omits essential acceptance conditions. Require the rolling 60-second decode floor , three qualification runs, cold-cache reset, no OOM/crash/repetition, and all six lab gates. Zero failures in 100 tool calls satisfies the spec but still gives an approximately 3% one-sided 95% upper bound on failure probability. The solve-rate test is also coarse: with four reference solves, all four are required; with five, four suffice. Publish these counts alongside occupied context and exact launch provenance.
0003-harness-interpretations.md 0003 — Harness interpretations of SPEC §5 (repetition, parse failures)0003 — Harness interpretations of SPEC §5 (repetition, parse failures)
Context: the T06a harness was exercised for 10 min against the operator's daily Qwen3.8-27B (EXL3 3 bpw, SGLang): 73 tool calls, 0 malformed tool calls, but 12 repetition hits — genuine reasoning loops (the same multi-paragraph block 3× inside one reasoning field) on sympy-21612 at 60–83k occupied tokens. Under §5 as written, the operator's everyday model fails stability.
Interpretations implemented by the harness:
Repetition, per field: the 64-token ×3 rule is evaluated within each generated field (reasoning, content, each tool call's arguments), not over the concatenation, because code drafted in reasoning and then emitted in a tool call trips the joined check without being a loop.
Parse failures, strict: besides invalid tool-call JSON and unparsed tool-call markup, every agent format error counts — including a response truncated at max_tokens (8192) with no tool call. Each response counts once; causes are split in detail.
Same-call streak (≥4 identical consecutive tool calls) resets at each task attempt.
Proposed (pending data, operator decides): keep §5 strict for this first pass and record all hits with causes; after T10/T20 produce real data, revisit whether "no repetition loops" should mean "no unrecovered loop" (a loop that ends the response by hitting max_tokens or ends the task) — not changed now.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
SPEC §5 amended: per-field repetition (cross-field diagnostic only), detector frozen for T10/T20, streak resets only on a fresh conversation (each task attempt starts a new mini-swe-agent conversation, so the harness already complies).
Three error categories: parse_failures (malformed tool calls only — the qualification criterion), output_truncations, agent_format_errors, all published.
No relaxation of "no repetition loops" now. Recovery and wasted-token figures to be published as supplementary data; any protocol revision later is prospective and does not re-grade earlier results.
Denominators always shown next to zero-failure claims.
0003-harness-interpretations.review.md 0003-harness-interpretations.review
AMEND. Keep the strict no-loop gate, but distinguish detector clarification from changes to eligibility. The strongest objection is that the harness simultaneously narrows repetition detection and broadens parse failures , while claiming to preserve §5.
Per-field repetition is defensible, but requires an explicit spec amendment. “Within one response” currently includes repetitions across fields. Adopt: “Within reasoning, content, or each tool-call argument field, independently, using the served model’s tokenizer.” Retain cross-field matches as diagnostics so the excluded cases remain auditable.
Do not label every agent-format error a tool-call parse failure. Invalid JSON and malformed tool markup qualify; an otherwise valid response exhausted at max_tokens without a tool call is a truncation or agent-protocol failure. Record tool_parse_failure, agent_format_error, and output_truncation separately. If all must disqualify, add that requirement explicitly to §5.
Resetting the same-call streak is appropriate only at an independent task attempt. Specify that a fresh agent conversation resets it; a retry continuing the same conversation does not. Otherwise attempt boundaries can conceal four consecutive identical calls.
The daily-model result does not justify relaxing the gate. Twelve genuine loops establish that this particular screening run fails the agreed criterion. They do not establish that the criterion is unsuitable for the two target models. The observed 60–83k occupancy also falls below the ≥128k requirement for “agentic-usable.”
Reject “unrecovered loops only” as the qualification alternative. Recovery does not erase wasted generation or latency. At exactly the specified decode floors, an 8,192-token response takes approximately 164 seconds for Flash-Next or 546 seconds for GLM , excluding prefill—27% or 91% of a ten-minute run. Publish recovery and wasted-token measurements as supplementary metrics while retaining zero detected loops for qualification.
The parse evidence is preliminary. Seventy-three calls falls short of the ≥100-call minimum. Under an independent binomial assumption, 0/73 failures still gives a one-sided 95% upper failure-rate bound of approximately 4.0% ; 0/100 gives 3.0% . Report the denominator without presenting zero observed failures as strong reliability proof.
Concrete alternative: ratify the field boundary and separate error categories before target comparisons; freeze that detector version for T10/T20; preserve each offending response and count affected responses separately from detector matches. Use later data to propose a prospective protocol revision, with existing results retaining their original qualification status.
0004-core2-quarantine.md 0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)0004 — Quarantine CPU core 2 (logical CPUs 2 and 18)
Evidence: two hard resets on 2026-10-03 (04:57 near-idle, 06:56 during T10 model load). Each next boot logged an uncorrectable machine check in bank MC5 (Execution Unit) — CPU:18 MC5_STATUS[-|UE|MiscV|-|PCC|TCC|SyndV] and CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV]. CPUs 2 and 18 are SMT siblings of physical core 2. No OOM, no GPU Xid, no kernel panic text: the core faulted and the platform reset. Likely cause: per-core Curve Optimizer/PBO undervolt too aggressive for that core, or core degradation (RMA case if BIOS is stock; BIOS 5101).
Decision (operator, option 2): take CPUs 2 and 18 offline for the whole campaign (resets at reboot; hygiene.sh check refuses to benchmark while they are online). Every record carries host.cpus_online/cpus_offline. llama.cpp and ik_llama launches use 15 threads. Then validate with tools/stability/cpu_check.sh (all-core AVX2, per-core single-thread boost bursts, all-core openssl) and look for new machine checks.
Consequence for results: numbers are for a 15-core 5950X. CPU-bound routes (GLM experts on CPU) are affected ~6 %; GPU/PCIe-bound trellis decode should not be. Any result produced before this decision does not exist (T10 never completed).
Check result (2026-10-03 07:25–07:40)
cpu_check.sh 300 20 180 on the 30 online CPUs: 0 machine checks, no reset (log: logs/cpu_check-core2-offline.log). Not conclusive for the near-idle failure mode; the hygiene check keeps the quarantine enforced for every run.
Revision (2026-10-03 10:30) — quarantine lifted with a tripwire
Operator BIOS check: PBO was Auto (ASUS stock); Curve Optimizer set to all cores, positive 0 (= no offset). So the CPU was effectively at stock when it faulted; a weak core 2 remains the leading hypothesis.
TARGET_CPUS="2 18" cpu_check.sh 300 20 180 with all 32 CPUs online: all-core AVX2, per-core bursts, 5 min sustained on CPU 2 and 18 each, 5 min on/off bursts on each, all-core openssl — 0 machine checks , no reset.
Not conclusive (both crashes came after hours of uptime). Core 2 is back online for full capacity; hygiene.sh check refuses to benchmark and re-offlines CPUs 2,18 if any machine check appears in the current boot; every record carries host.machine_checks_this_boot. llama.cpp/ik_llama launches back to 16 threads.
T10 ran on 15 cores (recorded in its host block). Trellis decode is PCIe-bound; one 16-core re-check is planned.
If it recurs at stock: keep core 2 offline for the campaign; RMA candidate.
Revision (2026-10-04 08:05) — permanent quarantine
Third fatal reset with core 2 online at stock: 2026-10-04 06:14, ~20.5 h into the boot, during the 16-core T10 re-check model load (CPU:2 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], logged at the 06:54 boot). Same pattern as the 2026-10-03 06:56 crash (Flash-Next load) after a long uptime.
Core 2 stays offline for the rest of the campaign, enforced at boot by local-ai-bench-quarantine.service (tools/box/install.sh) and by hygiene.sh check (CPU_QUARANTINE defaults to "2 18"). All llama.cpp/ik_llama launches use 15 threads. The 16-core T10 re-check is dropped; all results are for a 15-core 5950X.
Stock settings + repeated MC5 UE on one core: AMD RMA candidate (operator's call).
0005-ik-llama-glm-tool-parsing.md 0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)0005 — ik_llama.cpp does not parse GLM-5.3-Flash tool calls (commit 5bf8f0fe)
Finding (T22b, IQ2_M, 1 card): the model emits well-formed GLM tool calls (<tool_call>bash<arg_key>command</arg_key><arg_value>…</arg_value></tool_call>) but ik_llama's server returns them as plain content with empty tool_calls: lab tools gate fails; stability run 0 tool calls / 47 parse failures (unparsed_markup). ik_llama builds tool parsers from the chat template (common/chat-auto-parser*); mainline llama.cpp b11312 parses the same GGUF correctly (tools gate passed in T20b). No matching ik_llama issue/PR found (search 2026-10-04).
Speed is unaffected and recorded: decode 10.0 tok/s (mainline 9.6), prefill 151 lab / 203 TTFT-8k (mainline ~80).
Decision: keep the ik_llama route as speed evidence only (experiment-only, fails §5 tools); do not patch ik_llama in this campaign — single-card GLM is ~10 tok/s on every route, below the 15 tok/s gate, so tool parsing would not change the outcome. Phase 3 option: upstream bug report with the captured response (artifacts/runs/…ik_llama…stability/proxy/parse_failures/).
3-card follow-up (2026-10-04, GPU2-first order, decision 0006)
No MTP, --fit-margin 4096: lab decode 13.4 / 14.0, agentic 13.3 tok/s steady (best GLM decode on this box), prefill 185–283; tool calls still unparsed (68 agent-format errors), tools gate fails.
MTP, --fit-margin 7168 (4096 OOMed creating the MTP context): lab decode 10.2 (acceptance 51 %), agentic 15.6 whole-run before the server crashed mid-run. The extra margin for the draft context pushes experts to host RAM.
Route parked: experiment-only speed evidence. The most interesting number for 0xSero is 13–14 tok/s without MTP; a tool-parser fix in ik_llama plus an attested image would be needed before it could be a 3-card candidate.
0006-pcie-layout.md 0006 — PCIe lane layout: what can be reconfigured, and what it buys0006 — PCIe lane layout: what can be reconfigured, and what it buys
Operator question (2026-10-04): can the PCIe lanes be reconfigured to get better model performance?
Measured topology (lspci, nvidia-smi, T05a)
Device Attach Link Measured H2D GPU0 3090 04:00.0 X570 chipset, behind a Gen4 x4 uplink shared with the KC3000 1 TB NVMe (root + all model files), NIC, BMC, SATA, USB Gen4 x8 to the chipset 6.0 GB/s GPU1 3090 0b:00.0 CPU lanes, slot 2 (x8/x8 split) — drives the display Gen4 x8 not measured (display) GPU2 3090 0c:00.0 CPU lanes, slot 1 (x8/x8 split) Gen4 x8 13.5 GB/s 990 EVO Plus 4 TB CPU lanes (M.2_1, Gen4 x4) unused: holds another OS install
Already optimal and not a lever: every link trains at Gen4 (16 GT/s); Resizable BAR is on (32 GB BAR1); ASPM is disabled on every GPU link. AM4 has 24 usable CPU lanes (16 slots + 4 M.2 + 4 chipset uplink); bifurcation or risers cannot add lanes, so the only real choices are which card sits where and how many cards share the 16 slot lanes .
Where PCIe bandwidth limits each route (evidence from our runs)
Flash-Next trellis (experts streamed from host RAM to one GPU): decode is PCIe-bound. 0xSero's data: ~26 GB/s → 62.7 tok/s, 14–16 GB/s → 53.2. Ours at 13.5 GB/s: lab 49–55, kit C1 46.9, agentic 43. x16 should add ~15–25 %.
llama.cpp/ik_llama prefill with experts in RAM is PCIe-bound (CPU-resident weights are copied to a GPU for large batches). Evidence: on GPU2, raising -b/-ub 512 → 2048 lifted GLM IQ2_M prefill 85 → 200 tok/s (each copy serves 4× more tokens). And 3 cards prefilled slower than 1 card (158 vs 200), consistent with the copy going to CUDA0 = GPU0, the 6 GB/s chipset card.
llama.cpp decode with experts on the CPU is DDR4-bound, not PCIe-bound (GLM ~9.5 tok/s on 1 card, 12.3 on 3 cards, from less RAM-resident weight). PCIe changes nothing here.
Options
# Change Cost Expected effect S1 3-card llama.cpp with the CPU-attached GPU2 as CUDA0 (CUDA_VISIBLE_DEVICES=2,1,0 in the container) none 3-card prefill back to ≥ 1-card level (~200+), no decode change S2 Larger ubatch (-b/-ub 4096) on llama.cpp routes none (more VRAM for compute buffers) prefill up to ~1.5–2× again; watch VRAM (the IQ2_M 3-card run already OOMed on CUDA0) P1 One card alone in slot 1 at x16 (T12, already planned) operator moves cards Flash-Next decode +15–25 % (the only route to ≥ 50 headline at full context); llama.cpp prefill ~2× vs x8; GLM exact mode at the bandwidth its 12.6 tok/s reference used P2 Plug the monitor into GPU0 (chipset card) instead of GPU1 move one cable GPU1 + GPU2 (both CPU x8) free for 2-card runs without going headless; desktop stays usable during single/2-card work P3 Model files on the CPU-attached 990 EVO Plus instead of the chipset KC3000 needs the operator's OK (other OS lives there) removes model loads and Flash-Next n-gram NVMe reads from the chipset uplink that GPU0 also uses; small effect except for 3-card runs with GPU0 P4 2 cards on CPU lanes (x8/x8), third card removed or idle none (GPU1+GPU2) avoids the 6 GB/s card; less VRAM (48 GB) so more experts in RAM
Decision (proposed)
Run S1 and S2 now in the current layout (software only, attributable, one variable family each), on GLM IQ2_M: 3-card GPU2-first at ub2048 (vs the existing GPU0-first record), then 1-card GPU2 at ub4096 (vs ub2048).
Keep P1 (T12) as the main hardware lever; in the x16 session also rerun the best GLM llama.cpp 1-card config so the x8→x16 effect on prefill is measured directly.
Recommend P2 to the operator at the T12 visit (same visit, one cable). P3 only if the operator wants to free the 990 EVO Plus; not worth touching another OS for this campaign.
Publish this as a site methodology section with the before/after records.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted, plus operator rule
Bandwidth effects are hypotheses until measured. Expected-gain numbers above are estimates, not claims; x16 is the principal hardware hypothesis for Flash-Next, not a guarantee of qualification.
S1 is a fresh paired comparison, both headless, identical except card order: IQ2_M 3-card ub2048 with a 2 GiB --fit-target margin (the default-margin run OOMed on CUDA0 after 62 min), GPU0-first vs GPU2-first (CUDA_VISIBLE_DEVICES=2,1,0; the container sees the cards in PCI order). Every TTFT step now logs per-GPU PCIe rx/tx (nvidia-smi dmon -s t) to show which card receives the weight copies. Decode changes are reported, not assumed.
S2 : 1-card GPU2 ub2048 vs ub4096, both at 15 threads (the old ub2048 record ran 16 threads), peak VRAM and OOMs recorded.
P1 : rerun the best 1-card GLM llama.cpp config at x16 with the same bandwidth procedure (x8 control exists).
Operator rule (2026-10-04): GPU0 is used last. Single-card runs use GPU2; 2-card runs GPU2+GPU1; 3-card runs order the cards GPU2, GPU1, GPU0. Pending S1 evidence, all new multi-card launches use the GPU2-first order.
P2 offered to the operator at the T12 visit; P3 deferred (would need a §9 exception and touches another OS).
S1 result (2026-10-04 12:15–13:05 AEST) — card order, GLM IQ2_M 3-card ub2048, 2 GiB fit margin, headless
GPU0 first (control) GPU2 first (CUDA_VISIBLE_DEVICES=2,1,0) change TTFT prefill 8k / 32k 159 / 169 tok/s 240 / 242 tok/s +51 % / +43 % lab prefill 153 tok/s 218 tok/s +43 % lab decode C1 12.2 tok/s 11.8 tok/s −3 % PCIe rx during TTFT (avg / peak) GPU0 3.9 / 6.5 GB/s (link-saturated), others ~0.3 GPU2 5.7 / 14.3 GB/s, others ~0.5
The host→device weight copies for prefill go to CUDA0; putting a CPU-lane card there lifts prefill ~1.5×. Decode is within noise of unchanged (slightly lower). The GPU2-first config also finished a clean 10-min agentic run (11.3 tok/s, 24 tool calls, 0 parse failures, 0 loops) and passed the post-run gates without the OOM that hit the default margin. Rule confirmed: GPU0 last.
S2 result (2026-10-04 13:05–14:23 AEST) — ubatch, GLM IQ2_M 1 card GPU2, 15 threads
ub2048 (control) ub4096 change lab prefill 201 tok/s 288 tok/s +43 % TTFT prefill 8k / 32k 213 / 222 321 / 316 +51 % / +42 % lab decode C1 9.5 9.6 unchanged VRAM at ready 21.9 GB 22.9 GB +1 GB, no OOM in a 10-min agentic run
The 15-thread control matches the earlier 16-thread ub2048 record (9.5 / 200), so the core-2 quarantine costs nothing measurable on this route. Gain > 3 %, so the search continues: ub8192 on 1 card, and ub4096 on the GPU2-first 3-card config.
S2 step 2 (2026-10-04 16:13–16:30 AEST) — ub8192, 1 card GPU2
lab prefill 370 (+28 % vs ub4096), TTFT 384 / 404, but lab decode 9.1 (−5 %): the larger compute buffer makes --fit keep more experts in host RAM (VRAM at ready 22.2 GB). Prefill and decode now trade off, so the batching search stops here: ub4096 is the balanced setting (decode unchanged, prefill +43–51 %); ub8192 is the prefill-maximising variant, published as such.
3-card GPU2-first, ub2048 → ub4096 (16:30–16:58): lab prefill 218 → 258 (+18 %), TTFT 240 → 298 (+24 %), lab decode 11.8 → 11.2 (−5 %). Same prefill/decode trade-off as 1 card at ub8192; both published, no further steps.
0006-pcie-layout.review.md 0006-pcie-layout.review
AMEND. Proceed with S1, a bounded S2 test, and T12, but describe the bottlenecks and speedups as hypotheses. The strongest counter-argument is that the current runs change batching, weight placement, and GPU count; they do not isolate PCIe bandwidth.
Flash-Next’s +15–25% estimate is plausible; “PCIe-bound” overstates the evidence. The reference gain is only 17.9% despite bandwidth rising from 14–16 to 26 GB/s. A simple fixed-work Amdahl model attributes roughly 33–39% of baseline time to transfers, predicting 20–25% from doubling bandwidth. That model remains unverified across machines and contexts.
x16 does not establish qualification. Applying +15–25% to the measured agentic 43 tok/s gives 49.5–53.8 tok/s , and averages cannot prove every required rolling window clears 50. Replace “the only route to ≥50 headline” with “the principal hardware hypothesis”; qualification still requires the same config to pass occupied-context, prefill, quality, and stability requirements.
The prefill explanation is not established. Increasing batch size fourfold produced 2.35× throughput, from 85 to 200 tok/s. Transfer amortization, better GPU utilization, and execution overhead can all explain this. Likewise, 158 versus 200 tok/s across three versus one GPU does not identify CUDA0 as the cause. Replace “PCIe-bound” and “~2× at x16” with measured observations and an explicit hypothesis.
Make S1 a fresh paired comparison. Run GPU0-first and GPU2-first at ub2048 with identical weights, offload settings, tensor split, context, and cache state; both must be headless. Select devices by UUID , record their logical CUDA mapping, and verify which GPU actually receives the expert transfers. Reordering devices alone does not demonstrate the claimed routing change. Report decode changes rather than assuming none.
Bound S2 by memory and measurement. Compare one-card GPU2 ub2048 with ub4096 while holding the other settings fixed; record peak VRAM and OOMs. Remove the promised “1.5–2× again.” If both batch and ubatch change, identify the comparison as a batching-family change without attributing its effect specifically to ubatch.
Keep P1, with a matching x8 control. Run the exact same best one-card GLM configuration before and after the move, alongside bandwidth measurements under a defined, identical procedure. The 12.6 tok/s exact-mode reference already used x16 and remains 16% below the 15 tok/s floor ; matching its link does not imply registry eligibility.
P2 is useful, subject to the display rule. Moving the display to GPU0 enables display-free GPU1/GPU2 comparisons. Three-card tests still require logout and display: off; the x16 benchmark must also use a GPU that is not driving the display.
Defer P3 and public publication as proposed, but clarify scope. Storage contention has not been measured, and changing active-model storage would require an explicit exception to §9. Put the methodology and every before/after result in the private preview now ; public publication remains Phase 3 after operator and log review.
0007-glm-current-layout-finalists.md 0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs0007 — GLM current-layout finalists (T25a) and where the quality evaluation runs
Facts (all current-layout GLM records, 2026-10-03/04)
Route Cards Lab decode Lab prefill 10-min run (tool calls / parse failures / loops / solved) llama.cpp IQ2_M ub4096 1 9.6 288 27 / 0 / 1 / 1 of 2 llama.cpp IQ2_M ub2048, GPU2 first 3 11.8 218 24 / 0 / 0 / 1 of 2 llama.cpp IQ2_M ub2048, GPU0 first 3 12.3 158 26 / 0 / 0 / 1 of 2 (server later OOMed) llama.cpp IQ3_XXS ub2048 3 10.4 132 22 / 0 / 1 / 1 of 2 llama.cpp IQ4_XS ub2048, GPU2 first 3 8.3 156 25 / 0 / 0 / 1 of 2 llama.cpp Q2_K / IQ3_XXS 1 9.2 / 8.3 78 / 65 0–1 solved ik_llama IQ2_M (± MTP) 1 8.2–10.0 (MTP agentic ~12.6) ~150 0 tool calls: GLM tool calls not parsed (0005)
Every route is below the 15 tok/s registry gate on 1 card and on 3 cards (best 12.3). Decode is DDR4-bound: more cards only help by holding more experts in VRAM. Pending in the queue: IQ2_M 1-card ub8192 and 3-card ub4096 (prefill-only), ik_llama 3-card ± MTP (speed evidence only).
Proposal
No current-layout GLM finalist. §5 qualification requires decode ≥ 15 tok/s in every window, so the 3 × 10-min qualification runs would fail by construction. T25a records "no finalist: below the floor" for every current-layout config instead of running them. All configs stay published as documented experiments (§6 Phase 2).
One uncapped quality evaluation, of the best mainline weights, in the x16 session. Solve rate depends on weights, engine and sampling, not on the PCIe layout; the layout only changes how long the evaluation takes. So the T07 candidate evaluation of llama.cpp IQ2_M runs on the single x16 card next to the T23 reference (exact mode, or the highest-bit GGUF that fits if exact mode cannot load), with the identical task list, sampling and 60k-token budget. Each evaluation is estimated at 4–7 h (measured ~3–4k completion tokens per 10 min).
No uncapped evaluation for IQ3_XXS / IQ4_XS / Q2_K / ik_llama. They are slower than IQ2_M and cannot become candidates; their 10-min screening solve counts are published labelled "screening only". IQ4_XS only fits on 3 cards, so measuring it would cost ~6 h of current-layout time for an informational number.
Consequence: T12 (card move) can happen as soon as the current queue finishes (~3.5 h), not after ~1 day of current-layout quality runs. After the queue, IQ3_XXS and IQ4_XS are marked rejected and deleted (manifests kept) to make room for the x16 weights (Flash-Next 2.05/4.05 bpw: 63/108 GB; GLM exact EXL3: 125 GB).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Below-floor is measured, not inferred from lab C1: every current-layout llama.cpp config with a 10-min run has a measured rolling 60-s generation-only minimum below 15 (IQ2_M 3-card 11.0–11.4, IQ2_M 1-card 8.8–9.1, IQ3_XXS 3-card 9.9, IQ4_XS 3-card 7.8). Configs without a stability run (prefill-only S1/S2 controls, ub8192) are recorded as "qualification skipped: subfloor lab C1; rolling floor not measured". T25a closes with no current-layout finalist once the pending queue (ik_llama 3-card ± MTP, ub follow-ups) is in; ik_llama stays experiment-only (0005).
Quality evaluation is conditional. At x16: measure IQ2_M 1-card first; only a config that qualifies (§5 incl. the rolling floor, ≥ 100 cumulative tool calls, post-run gates) gets the uncapped T07 candidate evaluation. An optional quality study of a non-qualifying config is labelled informational. Quality results apply to the tested configuration; layout independence is not assumed.
Runtime estimate corrected: the 60k budget is per task; worst case 2.5–3.3 h per task, 12–33 h per evaluation (less when tasks submit early). Budget accordingly; this is another reason to evaluate qualifiers only.
Labels: IQ4_XS, IQ3_XXS (3-card), Q2_K: "rejected for speed, quality unmeasured". IQ2_M 3-card GPU0-first: OOM after 62 min (default fit margin). IQ2_M/IQ3_XXS repetition hits are published with the run.
Storage: keep IQ2_M (x16 rerun) and IQ3_XXS (highest-bit GGUF that fits 1 card = fallback reference if exact mode cannot load); delete IQ4_XS after the queue, manifests and hashes kept for reacquisition.
0007-glm-current-layout-finalists.review.md 0007-glm-current-layout-finalists.review
AMEND. Finish the queue and move to x16 without spending a day on current-layout quality runs. The strongest objection is that the proposal treats lab C1 as proof of failing a different metric , then schedules a “candidate” quality evaluation for a configuration that has not qualified.
“Fail by construction” is unsupported. Lab decode measures the first 30 seconds; §5 requires generation-only decode over rolling 60-second windows, ignoring the first minute. Best C1, 12.3 tok/s, is 18% below the floor and needs 22% uplift to reach 15. That warrants deprioritization, but not a claimed measurement of the floor. Use existing screening traces to establish a failing window; otherwise record “qualification skipped: subfloor lab C1; rolling-floor eligibility unverified.”
Make x16 quality evaluation conditional. §5 says “Finalists only.” First measure IQ2_M on x16; if promising, complete qualification, including ≥100 cumulative tool calls and all six post-run gates. Run the uncapped comparison only if it qualifies. If it remains below 15, finish it as a documented experiment under Phase 2. Any optional quality study should be explicitly labelled informational.
Layout independence is a hypothesis, not a guarantee. Fixed weights and token budgets remove much of the speed bias, but device placement can change numerical execution and sampled trajectories; timeouts, OOMs and truncation can also affect outcomes. The x16 evaluation directly supports the tested x16 configuration . Transfer its quality result to another layout only with matching engine, settings and evidence of equivalent behavior.
IQ2_M is the speed leader, not established “best weights.” The 10-minute results cannot rank quantization quality. IQ4_XS could be slower yet substantially more accurate—a relevant possibility given §12’s sub-4-bit quality risk. Skipping its uncapped evaluation is reasonable because it has no demonstrated eligibility; label it rejected for speed , with quality unmeasured.
The runtime estimate needs its budget scope stated. At 3–4k completion tokens per 10 minutes, exhausting 60k tokens per task takes 2.5–3.3 hours per task. Across 5–10 tasks, that is 12.5–33 hours per evaluation , or 25–67 hours for candidate plus reference , assuming similar throughput. Early completion can reduce this. If 60k means an aggregate budget, specify the per-task allocation; “4–7 hours” currently lacks a clear basis.
Keep T25a provisional until the queue finishes. Pending ik_llama 3-card/MTP measurements could change the speed ranking, although parsing and image eligibility remain blockers. Existing IQ2_M repetition and GPU0-first OOM evidence also need explicit failure labels. Faster decode alone does not make these configurations finalists.
Treat “DDR4-bound” as a working diagnosis. The table does not isolate memory bandwidth from PCIe, placement, batching or engine overhead. The one-card 9.6 tok/s result needs 56% uplift to reach 15, so an x16 rescue is uncertain. Measure it before committing to expensive quality work.
Concrete replacement: finish pending measurements → close current-layout records with measured failures or justified skips → move cards → measure and qualify x16 routes → evaluate qualifying candidates against the specified reference. Delete rejected weights after preserving hashes and immutable references, while reserving or budgeting reacquisition of the highest-bit GGUF fallback. Keep results private until Phase 3.
0008-flashnext-3card-llamacpp.md 0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate0008 — Flash-Next on 3 cards with llama.cpp: tune it as a separate 3-card candidate
Evidence (T11, 2026-10-04 15:38 AEST, IQ4_XS, GPU2,GPU1,GPU0 order, 2 GiB fit margin, -b 2048 / -ub 512 defaults)
Lab: all six gates pass , decode C1 57.1 / 56.9 tok/s (before / after the agentic run), lab prefill 608 / 605.
TTFT prefill: 987 tok/s at 8k, 852 at 32k.
10-min agentic run: 70 tool calls, 0 parse failures, 0 loops, 1 of 2 solved, whole-run decode 45.4, lowest 60-s window 38.6 (fails the 50 floor; decode falls as the context grows).
VRAM at ready 65.5 GB of 72; weights 97.7 GB, so ~30+ GB (incl. the n-gram embedding table) stays in host RAM.
For comparison, the 1-card trellis/SGLang baseline at x8: lab 49–55, kit C1 46.9, agentic 43.
Per SPEC §6 a 3-card config is a separate, clearly labelled recipe; llama.cpp routes are judged by the registry lab and §5 (the kit's SGLang-only panel cannot score a GGUF, so it is "kit band not measurable").
Proposal (current layout, before T12; one variable family per step, stop rule 3 × < 3 %)
Re-download IQ4_XS (deleted at 16:13 per plan; manifest kept, 97.7 GB, ~15 min).
Prefill : -b/-ub 2048, then 4096 (GLM showed +135 % then +43 %; target lab prefill > 1000).
Decode floor : the floor fails late in the run as context grows. One quant step down to IQ3_M (93.2 GB) or IQ3_XXS (88.0 GB) puts more experts in VRAM; run the better prefill setting from step 2 on it. Quality cost is unmeasured by the kit for GGUF; report solve counts and parse failures, label "quality: screening only".
If a config reaches lab decode ≥ 50, lab prefill > 1000 and holds the 50 floor in a 10-min run, it becomes a 3-card finalist: §5 qualification (3 runs incl. cold and max context, ≥ 100 tool calls) before T12.
Disk: the x16 downloads (296 GB) wait until this route is done and its rejected GGUFs are deleted.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Route is experiment-only unless an accepted scoring path establishes the Flash-Next kit band for a GGUF; kit band recorded as unavailable, registry/headline eligibility unestablished. The T11 config is a failed screening config (floor −29.5 %, lab prefill −65 % vs targets), reported per constraint.
Batching steps, predeclared, judged by official lab prefill (decode and VRAM regressions checked): A -b 2048 -ub 2048, B -b 4096 -ub 4096, both vs the existing -b 2048 -ub 512 record. (GLM evidence says the ubatch sets the host→GPU copy amortisation, so b and ub move together.)
Before any quant change: decode vs occupied context from the existing T11 agentic trace (per-request generation speed against prompt tokens). IQ3_M is screened only if that shows a residency/transfer limit, not attention cost; its step is judged by the minimum rolling-window decode at matched occupancy. Expert/n-gram placement and memory use recorded from the server log.
Qualification only for a speed-passing config, naming its maximum occupied context (≥ 4k headroom), cold resets, post-run gates, ≥ 100 tool calls cumulative. Phase timebox kept; x16 is not deferred beyond this route's steps.
Decode vs occupied context (T11 3-card agentic trace, 55 requests, generation-only tok/s per request)
prompt tokens per-request decode 1.5k–6k (django task) 35–51, typical ~42 6k–20k 30–49, typical ~40 20k–31k 30–41, typical ~38 31k–39k 28–38, typical ~33
Two separate gaps to the 50 floor: a base gap (~42 tok/s already at 2–6k tokens, vs lab C1 57 measured over one long generation; agentic requests are short, 70–600 tokens) and a context slope of roughly −20 % from 5k to 39k. One quant step (IQ3_M, −4.6 % of bytes; IQ3_XXS −9.9 %) cannot close a 25–30 % gap even if every saved byte moved experts to VRAM. IQ3_M step dropped ; the predeclared batching steps still run (prefill evidence, cheap). After them the route closes as experiment-only and its GGUF is deleted for the x16 weights.
Batching result (2026-10-04 18:00–18:23 AEST) — route closed
-b / -ub lab decode C1 lab prefill TTFT 8k / 32k 2048 / 512 (T11 record) 57.1 608 987 / 852 2048 / 2048 44.3 (−22 %) 717 (+18 %) 1067 / 931 4096 / 4096 35.0 (−39 %) 713 980 / 881
Larger ubatches buy little prefill and cost a lot of decode: the bigger compute buffers make --fit move experts to host RAM. No setting reaches lab prefill > 1000 while keeping decode ≥ 50, and the agentic floor already failed at the default. Route closed: experiment-only , best config = the default-batch T11 record (all six gates, lab 57.1, agentic floor 38.6). GGUF deleted for the x16 weights (manifest kept).
0008-flashnext-3card-llamacpp.review.md 0008-flashnext-3card-llamacpp.review
AMEND — retain this as a separate 3-card experiment, but remove the promised path to registry eligibility until quality is measurable. The strongest counter-argument is contractual: §5 explicitly requires the Flash-Next kit band for both the headline and registry PR. “Kit band not measurable” cannot establish a pass. §12’s lab-only tooling note creates an ambiguity; it does not explicitly waive that requirement.
T11 is a failed screening config despite passing all six lab gates. Its worst window needs 29.5% improvement to reach 50 tok/s; lab prefill needs approximately 65% improvement to exceed 1000. Lab C1 at 56.9–57.1 does not compensate for either failure. Report these as separate constraints.
The prefill experiment is reasonable; the GLM gains are insufficient evidence for its expected payoff. Specify exact settings rather than “-b/-ub 2048, then 4096.” A concrete sequence is baseline -b 2048 -ub 512, then -b 4096 -ub 512, then -b 4096 -ub 1024, measuring identical occupied contexts and fresh prompts. Select using official lab prefill, and check for decode or memory regressions.
The quantization hypothesis needs a residency measurement. IQ3_M saves 4.5 GB (4.6%) ; IQ3_XXS saves 9.7 GB (9.9%) . Against roughly 30 GB remaining on the host, those represent at most approximately 15% / 32% reductions , assuming every saved byte reduces host residency. They do not prove that more active experts become GPU-resident or that transfers shrink proportionally. Record expert placement, n-gram placement, and actual memory use.
“Decode falls as context grows” establishes correlation, not the bottleneck. Before changing quant, report generation speed against occupied context, cache state, and GPU placement. Repeat controlled requests at representative early and late occupancies using IQ4_XS. Otherwise a smaller quant could obscure a context-dependent attention or transfer limitation.
Solve counts cannot replace the required quality comparison. One of two solved tasks is weak screening evidence, especially under a wall-clock cap where faster configurations receive more generation opportunities. Keep “quality: screening only,” but explicitly mark the kit band unavailable and registry/headline eligibility unestablished for every GGUF lacking valid panel scores.
Qualification must name the context it qualifies. Step 4 needs a claimed maximum occupied context, ≥4k generation headroom, the defined cold-cache resets, and post-run lab gates for each qualification run. “Agentic-usable” additionally requires ≥128k occupied tokens. Count ≥100 tool calls across screening, qualification, and any extra runs, as specified; publish truncations and agent-format errors too.
Make the search bounded and the stop rule meaningful. Predeclare which metric determines each step’s “gain”: official lab prefill for batching, minimum rolling-window decode at matched occupancy for quantization. A gain in one metric cannot excuse losing another acceptance criterion. Keep the phase’s 2–3-day timebox; disk sequencing alone should not defer x16 indefinitely.
Concrete revised decision: restore pinned IQ4_XS; run the two explicit batching changes; establish the context-dependent slowdown; screen IQ3_M, then IQ3_XXS only if justified. Run full stability qualification only on a speed-passing configuration. Unless the existing accepted scoring path establishes the kit band, publish the route as experiment-only , preserve its records, delete rejected weights under §9, and proceed to x16. Every 3-card record must include GPU UUIDs and display: off.
0009-no-x16-session.md 0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)0009 — No x16 session: Phase 1 and T23 continue in the current layout (operator decision)
Operator, 2026-10-04 19:15 AEST: skip the x16 test; the expected difference is judged marginal. T12 (card move) and T05b are cancelled. The operator's call stands; the estimate it overrides is recorded for the reader: 0xSero's reference gives +17.9 % decode for 14–16 → ~26 GB/s, and our x8 baseline (lab C1 49–55, kit C1 46.9, agentic 43, floor not held) sits just under the 50 tok/s target, so reaching it now depends on tuning at x8.
Re-plan (single card = GPU2, CPU-lane x8, 13.5 GB/s; GPU1 measured identical)
T13 PR #144 rerun at x8 against image f77c72f6 (lab, kit sweep with early exit off, TTFT, panel, 10-min run). Proof for 0xSero is "x8 on this box"; whether he accepts it is asked in Phase 3.
T15 brought forward: the main recipe on EXL3 2.05 and 4.05 bpw, only the weights changed. At x8 the expert bytes per token are the bottleneck, so 2.05 bpw is the most likely route to ≥ 50 sustained; the kit panel decides whether it stays in the quality band (required for the headline and registry PR). 4.05 bpw is the quality anchor.
T23 GLM exact mode on GPU2 at x8 (route test + quality reference; may not load: ~119 GB pinned on ~125 GiB).
T14a/T14b grid on the best quant from 1–2 (expert cache GB × context × KV format; then prefill chunk and n-gram tier), designed as its own decision once 1–3 are in. T16 qualification and headline follow on the winner.
GLM IQ2_M x16 rerun dropped; IQ3_XXS kept as the fallback reference.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Recorded as a deliberate restriction of the search, not evidence that x16 is marginal. Gap to target at x8: kit C1 46.9 → 50 needs +6.6 %; agentic 43 → 50 needs +16.3 %; there is currently no qualifying Flash-Next result.
Order: T13 at 3.05 → matched 2.05 / 4.05 screens (same launch, same context) → kit panel as a filter (top-1 ≥ 0.987, KL ≤ 0.0013) before any headline time → bounded tuning of in-band candidates, 3.05 included . Out-of-band results are published with kit eligibility, never as the registry path. "Bytes per token" is a hypothesis; transfer/cache evidence is recorded where the engine exposes it.
Acceptance unchanged: every eligible rolling 60-s window ≥ 50 and lab prefill > 1000 on the same config at the claimed occupied context. If no x8 config meets speed and quality together, that outcome is published as is.
T23 at x8: exact mode may be the GLM quality reference even below 15 tok/s (not a recipe); peak host memory and any load failure recorded (~110.8 GiB pinned vs ~125 GiB MemTotal). If it cannot load, the reference is the highest-bit GGUF that actually fits one card here, determined by a load test, not assumed to be IQ3_XXS.
Timebox and the 3 × < 3 % stop rule stay. Cancelled tickets (T12, T05b, x16 parts of T13/T14/T23) are recorded as skipped with the operator's reason.
0009-no-x16-session.review.md 0009-no-x16-session.review
AMEND. Respect the operator’s cancellation of x16, but describe it as a deliberate restriction of the search—not an evidence-backed conclusion that the gain is marginal. The strongest counter-argument is that the cited 17.9% uplift could change eligibility , whereas lower quantization risks losing the required quality band.
The target gap is material and measurable. Kit C1 46.9 → 50 requires +6.6% ; agentic average 43 → 50 requires +16.3% , and reaching that average would still not prove the rolling floor. Applying the reference uplift illustratively gives 55.3 and 50.7 tok/s , respectively. These are scenarios, not predictions for this machine or workload.
Retain 3.05 bpw as a tuning candidate. The proposed sequence risks selecting an out-of-band 2.05 model or a slower 4.05 model before exploring the original quant. Concrete alternative: T13 at 3.05 → matched 2.05/4.05 screens → quality filter → bounded tuning of qualifying candidates, including 3.05 .
Treat “expert bytes per token are the bottleneck” as a hypothesis. Neither the bandwidth measurement nor the quoted speeds establishes how much decode time is exposed transfer, compute, synchronization, or cache misses. Moving 3.05 → 2.05 bpw reduces nominal weight bits by 32.8% ; under a purely weight-transfer-limited model that suggests about 49% more throughput. Metadata, cache residency and other costs can substantially change that result. Record transfer/cache evidence where available before assigning causality.
Use the panel early as a search filter. Require top-1 ≥ 0.987 and KL ≤ 0.0013 before spending headline qualification time on a quant. Publish outside-band results and retain their kit eligibility, but do not make them the sole path toward the registry target. Keep occupied context and other settings matched for the initial weight comparisons.
Preserve the actual acceptance test. Lab C1 over 30 seconds, a kit score, and a whole-run agentic average cannot establish a sustained floor. The winner needs every eligible rolling 60-second window ≥ 50 , all qualification requirements, and lab prefill > 1000 on that same config , at the claimed occupied context. The existing “floor not held” evidence means there is currently no qualifying Flash-Next result.
T23 is useful at x8, but distinguish reference eligibility from recipe eligibility. Exact mode may serve as the GLM quality reference even below 15 tok/s ; it then remains ineligible for a registry recipe. 119 decimal GB ≈ 110.8 GiB , leaving roughly 14.2 GiB against the stated 125 GiB MemTotal before runtime and system costs. That is plausible but unproven headroom; record peak host memory and any load failure.
IQ3_XXS cannot automatically become the GLM quality reference. The spec requires the highest-bit GGUF that fits when exact mode cannot load. Keep IQ3_XXS as a performance fallback, but separately determine the fitting quality reference and run the identical-task, identical-budget, uncapped comparison required by §5.
Make the amendment auditable and bounded. Update §6, §11 and the x8 risk discussion; record cancelled tests as skipped with the operator’s reason. Keep the phase timebox and diminishing-return stop rule. If no x8 config meets speed and quality together, publish that outcome explicitly rather than relaxing acceptance. The revised deliverable is the best qualifying result found within the x8 search .
0010-cpu-boost-off.md 0010 — CPU boost off for the rest of the campaign0010 — CPU boost off for the rest of the campaign
Evidence (2026-10-04 19:48 AEST): hard reset during the T13 Flash-Next SGLang model load with core 2 already offline. The next boot decoded CPU:1 MC5_STATUS[-|UE|MiscV|AddrV|PCC|TCC|SyndV], "uncorrected error caused a data fabric sync flood" — the same bank, status and syndrome as the core-2 faults (0004), now on core 1 . Tally of fatal resets: 5, of which 3 during a Flash-Next SGLang load (expert repacking: heavy multi-threaded host work) and 2 after long uptime. Temperatures at idle are normal (Tctl ~56–60 °C). A second core failing the same way points to the CPU (or its voltage under boost transients) rather than one bad core.
Decision: disable Precision Boost from Linux (/sys/devices/system/cpu/cpufreq/boost = 0, all cores capped at the 3.4 GHz base clock, lower voltage), applied at boot by local-ai-bench-quarantine.service and immediately on 2026-10-04 20:12. Core 2 stays offline. Every record now carries host.cpu_boost and host.cpu_max_mhz.
Consequence: CPU-bound work (GLM experts on CPU, model loads) gets slower; PCIe-bound trellis decode should not. Records before 20:12 ran with boost on and are labelled by that field's absence (= boost on). If faults continue with boost off, the CPU is not usable for unattended runs and the remaining work pauses for the operator (RMA).
0011-flashnext-x8-tuning-grid.md 0011 — Flash-Next tuning at x8 (T14): grow the expert cache0011 — Flash-Next tuning at x8 (T14): grow the expert cache
Evidence
T15 is not runnable: the offload engine's HostExpertStore only supports 3-bit experts (offload_store.py: if bits != 3: raise NotImplementedError), so 2.05 and 4.05 bpw crash at load in both images. Only 3.05 bpw remains (the CPU-expert fallback, SGLANG_EXL3_MOE_OFFLOAD=cpu, is far slower and not pursued).
T13 (PR #144 launch, f77c72f6, x8, boost off): lab C1 54.5–55.6, lab prefill 2,433–2,596, kit C1 48.2 (32k: 49.9), kit panel in band (top-1 0.9887, KL 0.00097), agentic whole-run 44.1, min 60-s window 37.9 (floor 50 fails).
Agentic per-request decode is flat across context (median 43.7 / 43.8 / 41.7 tok/s at <8k / 8–20k / 20–40k), so context length is not the limiter; the gap to lab C1 (55) is consistent with expert-cache misses on coding traffic (each miss streams experts over x8 PCIe). Hypothesis, to be tested by changing the cache size.
VRAM budget at ready (server log): expert cache 11.08 GB (auto-fit: free 19.82 − staging 0.48 − reserve 8.26), KV pool 5-bit 1.77 GB for 210k tokens, 3.23 GB still free after CUDA graphs . Context length is therefore a weak lever (the whole KV pool is 1.77 GB); the reserve is the strong one.
Grid (base = PR #144 launch on the latest main-built image a21efea6, GPU2 x8; one variable per step)
Step Change Expected expert cache R1 SGLANG_EXL3_EXPERT_CACHE_RESERVE_GB 8.26 → 6.0~13.3 GB (+20 %) R2 reserve → 5.0 (only if R1 is stable and gains ≥ 3 %) ~14.3 GB (+29 %) N1 SGLANG_EXL3_NGRAM_TIER nvme → pinned (32.6 GB pinned RAM) on the best Rn-gram reads from RAM G1 --max-running-requests 1 --cuda-graph-max-bs-decode 1 (single-stream agentic) on the best, then re-fit the reservefrees graph memory
Metric per step: minimum rolling 60-s window and median per-request decode in the 10-min agentic run (the floor), with lab C1 and lab prefill checked for regressions. Every step runs lab,ttft,stab; the kit panel is re-run on the winner only (cache size does not change the math). Stop rule: 3 consecutive steps < 3 % on the floor metric. If no step lifts the minimum window to ≥ 50, Phase 1 closes with "not reached at x8" , publishing the best config.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Cache misses are a hypothesis: the floor needs +31.9 % (26.4 → 20.0 ms/token); a 20–29 % larger cache need not deliver that. Expert-cache hit/miss and transfer stats are recorded if the engine logs them, else marked unavailable.
Order: R0 (unchanged PR #144 launch on a21efea6) → R1 reserve 6.0 → G1 single-stream graphs on R1 → reserve refit only if peak VRAM (prefill and max-occupancy) shows room → N1 pinned n-grams with host-RAM headroom recorded. R2 (5.0) is not run blind: measured headroom after R1 would be ~0.97 GB, after R2 ~−0.03 GB.
max-running-requests 1 is a recipe concurrency constraint, stated as such.
Floor = minimum rolling 60-s generation-only window after the first 60 s (harness definition); window coverage reported. One 10-min run screens; the winner then needs full §5 qualification, kit band, and ≥ 128k occupied for "agentic-usable". Closure wording: "50 tok/s floor not reached in the tested x8 configurations".
R0 result and re-plan (2026-10-04 22:42 AEST)
R0 (a21efea6, PR #144 launch unchanged): load 260 s, lab C1 55.8, lab prefill 2,419, kit C1 47.4 (32k 48.6), TTFT 2,009 / 2,681, panel in band (top-1 0.9897, KL 0.00097) — then the 10-min agentic run crashed: CUDA OOM . After serving starts, Triton kernels load lazily (log: "device-loaded after serving started … free 0.83 → 0.09 GiB") and consume the 3.23 GB that looked free at ready. So the 8.26 GB reserve is needed on this image; shrinking it cannot grow the expert cache safely. (f77c72f6 survived the same agentic run at the same reserve.) Re-plan: R1 (6.0) is already running and is kept as evidence; R1+G1 dropped . Next: R0 repeat (is the OOM reproducible?), G1 at reserve 8.26 (single-stream frees graph and request memory → headroom), N1 pinned n-grams.
R1 (reserve 6.0, 22:42): load OK, then CUDA OOM at the first lab request — reserve cannot go below 8.26.
R0 repeat, agentic run only (22:58–23:22): no OOM ; 71 tool calls, 0 parse failures, whole-run 43.3, min window 39.7; lab after 57.4 / 2,537. The first R0's OOM is intermittent (it followed lab, kit sweep, TTFT and panel on the same server). Reported as an a21efea6 stability risk for PR #144; frequency unknown from 2 runs.
G1 single-stream at reserve 8.26 (23:22): min window 40.1 (+1 % vs R0 repeat), whole-run 43.1, lab 54.6–55.6.
N1 pinned n-grams (23:48): min window 40.9 (+3.1 %), whole-run 44.6, lab C1 53.7 / 51.3 (lower), prefill unchanged.
Closure (2026-10-05 00:11 AEST)
Stop rule met: R1 no gain (OOM), G1 +1 %, N1 +3 % (within run-to-run spread: two R0 runs and T13 span 37.9–39.7). Phase 1 result: the 50 tok/s floor was not reached in the tested x8 configurations. Best measured agentic floor ~40–41 tok/s (−18 to −20 %); lab C1 55–57, lab prefill ~2,400–2,600, kit panel in band. Best config for publication: the PR #144 launch (image a21efea6, or f77c72f6 which had no OOM in its run). No T16 finalist; T18 registry recipe not prepared; the PR #144 rerun evidence and the kit submission (with this tuning table) are prepared locally for Phase 3.
0011-flashnext-x8-tuning-grid.review.md 0011-flashnext-x8-tuning-grid.review
AMEND — growing the cache is a reasonable experiment, but the evidence does not establish cache misses as the bottleneck. Flat decode across context only weakens the context hypothesis. The floor needs +31.9% (37.9 → 50 tok/s); a 20–29% larger cache does not imply that speedup. The strongest counter-argument is that coding traffic accesses a broad expert set, so extra capacity may barely reduce misses.
Establish R0 on the new image first. T13 used f77c72f6; the grid uses a21efea6. Comparing directly changes both engine and reserve. Run the unchanged launch on the new, full digest-pinned image, then compare every tuning step against a baseline on that image.
R2 appears to exceed the measured memory budget. Assuming unchanged allocations and consistent units, R1 consumes another 2.26 GB , leaving approximately 0.97 GB after graphs; R2 consumes 3.26 GB , leaving approximately −0.03 GB . “Stable R1” cannot establish that R2 fits. Record peak VRAM during prefill and the claimed occupied-context run before reducing the reserve further.
Concrete alternative: R0 → guarded R1 → G1 → reserve refit → N1. Move graph reduction ahead of R2 when memory is tight, and measure each change separately. Treat max-running-requests=1 as an explicit recipe concurrency constraint; verify that the representative agentic workload fits it. For pinned n-grams, record host-memory headroom and end-to-end performance.
Test the mechanism alongside speed. Record expert-cache hit/miss counts, transferred bytes and transfer stalls if exposed; otherwise mark them unavailable and keep the attribution hypothetical. Going from 37.9 to 50 requires reducing generation time from 26.4 to 20.0 ms/token —a 6.4 ms/token saving. Establish whether expert transfers plausibly account for that much time.
Specify the floor calculation exactly. Ignore the first 60 seconds and calculate rolling windows using generation time only, excluding prefill, tools and idle time. Report window coverage and generation duration. Median request speed and whole-run average are diagnostics; neither establishes the floor.
Separate screening from acceptance. One 10-minute run can select candidates. The winner still needs the three §5 qualification runs—including fully cold and claimed maximum occupied context—≥100 tool calls across its runs , zero parse failures, repetition checks, all six lab gates afterward, and the kit quality band. “Agentic-usable” additionally requires ≥128k occupied tokens .
Narrow the closure claim. Close under the phase timebox or the specified three consecutive measured sub-3% gains, recording skipped branches explicitly. Publish “50 tok/s floor not reached in the tested x8 configurations” rather than implying x8 is exhausted. Prepare results and submissions locally; public publication remains Phase 3.
0012-glm-outcome.md 0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)0012 — GLM-5.3-Flash outcome (T24, T25a, T25b, T26)
Result: no GLM-5.3-Flash configuration on this box reaches the registry's 15 tok/s gate; every route is a documented experiment, and no GLM registry PR is prepared. All numbers: lab C1 decode / lab prefill, current layout.
Route Cards Decode Prefill Tools Classification llama.cpp b11312 IQ2_M, ub4096 1 9.6 288 pass, 0 parse failures experiment (1 card < 15) llama.cpp IQ2_M, ub2048, GPU2-first 3 11.8 218 pass experiment (3-card, < 15) llama.cpp IQ3_XXS / IQ4_XS 3 10.4 / 8.3 132 / 156 pass rejected for speed, quality unmeasured llama.cpp Q2_K / IQ3_XXS 1 9.2 / 8.3 78 / 65 pass rejected for speed ik_llama 5bf8f0fe IQ2_M (no MTP) 3 13.4–14.0 185 fails (tool calls unparsed, 0005)experiment-only, unattested image ik_llama IQ2_M + MTP 1 / 3 8.2–12.6 ~150–166 fails experiment-only; OOM / server crash 0xSero glm53-flash-offload, exact mode (x8) 1 7.4–7.5 720–739 pass, 0 parse failures quality reference; route below gate at x8
T25a: no current-layout finalist (decision 0007; rolling-floor failures measured in every 10-min run).
T24: the uncapped candidate-vs-reference comparison runs only for finalists, so it is not run. Exact mode loads on this box (~120 GB host RAM, 23.3 GB VRAM) and stays available as the reference if a future config qualifies.
T25b: 1-card routes < 15 → experiment; 3-card routes < 15 → no separate recipe; ik_llama → experiment-only until a tool-parser fix and an attested local-ai-images build (operator to raise with 0xSero).
T26: not applicable.
Hardware context for the reader: dual-channel DDR4-3200 (~16.6 GB/s host memcpy measured), x8 PCIe (13.5 GB/s), 15 cores at base clock (decisions 0004, 0010). Fast mode (0xSero's 28.9 tok/s route) needs ~238 GB RAM: not possible on AM4.
0013-flashnext-oom-fix.md 0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)0013 — Fix the intermittent CUDA OOM of the PR #144 launch (image a21efea6)
Operator request (2026-10-05): rerun the Qwen3.8 (Flash-Next) test and fix the stability issues.
Diagnosis (R0 first run, 2026-10-04 12:32 UTC)
The server died in a prefill: offload_runtime._prefill_parts_compact → `torch.AcceleratorError: CUDA error: unknown error, 29 s after Triton kernel '_row_logsumexp_kernel' device-loaded after serving started (free device mem: 0.00 GiB)`. Earlier in the same lifetime, during the 166k-token context gate, PyTorch logged repeated expandable_segments: memory mapping failed with OOM (free 10 MB of 25.3 GB).
At ready the engine reports 3.23 GB free, but kernels that load lazily after serving starts, plus prefill activations for 12,288-token chunks, consume it. Whether a run crashes depends on which kernels load late and the size of the next prefill: 1 of 3 lifetimes of the unchanged launch crashed; reserve 6.0 crashed at once.
Plan (GPU2 x8, full flow lab → kit sweep → TTFT → panel → 10-min agentic, the sequence that crashed)
Every flow now samples VRAM every 0.5 s (logs/vram/*.csv, summary = minimum free MiB): headroom is measured directly instead of waiting for random crashes.
Runs: baseline (unchanged launch) → A reserve 8.26 → 9.26 GB → B --chunked-prefill-size 12288 → 8192.
Pick the fix with the larger minimum free VRAM and the smaller speed cost (lab C1, kit C1, prefill, agentic floor), then repeat it twice (3 clean lifetimes) before calling it fixed. If neither leaves ≥ 1 GB minimum free, combine them.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Framed as memory-pressure mitigation , not proof the intermittent failure is gone: 3 clean lifetimes would still occur with probability 29.6 % at the old 1-in-3 crash rate (95 % upper bound on crash probability 63 %). The main evidence is the measured VRAM headroom margin; the diagnosis stays a hypothesis (the final unknown error is not itself a proven OOM). Allocator peak/reserved stats are not exposed outside the engine → recorded as unavailable.
Configs, one variable each: baseline (reserve 8.26, chunk 12,288), A (9.26, 12,288), B (8.26, 8,192); (9.26, 8,192) only if neither single change leaves margin. Actual expert-cache GB recorded from the server log for each.
Selection: Flash-Next has no qualifying config at x8 (floor not reached, decision 0011), so the target-based rule cannot pick a winner; the rule is **the config with the largest minimum free VRAM whose lab C1, kit C1, lab prefill and agentic floor stay within 3 % of baseline**; ties → baseline settings. ≥ 1 GB minimum free is an engineering margin, not a spec criterion.
Image provenance: a21efea6 = ghcr.io/0xsero/sglang-exl3-flashnext@sha256:a21efea6fe15ccc6a8774b1ca2a4c4104388cdcca8f1463645c02c0f8aba0163, local-ai-images run 37113268282 on main, trellis-serve b1e74af; it replaces the PR's branch-built digest per 0xSero's PR #144 comment of 2026-10-03.
Note: the queue skipped the baseline line (identical text already in campaign.done from 2026-10-04); re-added with an explicit step list, so it runs after A and B.
Results (2026-10-05, GPU2 x8, same full sequence lab → kit sweep → TTFT → panel → 10-min agentic)
Config Min free VRAM (0.5 s samples) Lab C1 Kit C1 Lab prefill Agentic floor / avg Outcome baseline (8.26, 12,288) 2 MiB (10:41, kit sweep)49.9 47.8 2,419 — / 43.0 crash in agentic run, 6 tool callsA (9.26, 12,288) 912 MiB 48.1–49.7 43.7 2,408–2,564 37.4 / 41.1 clean, 71 tool calls B (8.26, 8,192) 1,060 MiB 49.5–54.9 48.2 2,131–2,237 39.5 / 43.9 clean, 64 tool calls
Baseline crash: Triton kernels loaded after serving started at "free device mem 0.00 GiB" (alloc_extend_kernel, _row_logsumexp_kernel, _sparse_gqa_chunk_prefill), then CUBLAS_STATUS_INTERNAL_ERROR in cublasGemmEx (cuBLAS could not get workspace). The unchanged launch has now crashed in 2 of 4 lifetimes; it runs the card to ~0 free VRAM. Chosen: B — the most headroom, decode unchanged (kit C1 +1 %, floor −0.5 %), prefill −8 to −12 % (still ~2,100+, over twice the 1,000 target). A costs 8–13 % decode (smaller expert cache). Two more B lifetimes queued.
Confirmation (2026-10-05 10:59–12:29 AEST): fix B, 3 lifetimes
Lifetime Step order Min free VRAM Lab C1 (before / after) Kit C1 Agentic floor / avg, tool calls Outcome B1 lab, kit, ttft, panel, stab 1,060 MiB 54.9 / 49.5 48.2 39.5 / 43.9, 64 clean B2 lab, kit, ttft, panel, stab 1,060 MiB 56.4 / 53.5 47.0 38.8 / 43.2, 62 clean B3 lab, ttft, panel, kit, stab 1,060 MiB 55.0 / 45.0 48.1 40.4 / 44.0, 61 clean
Result: memory-pressure mitigation confirmed for this sequence. The minimum free VRAM is identical in all three lifetimes (1,060 MiB), so the peak is deterministic and the margin is real, against ~0 MiB and a 2-in-4 crash rate for the unchanged launch. 187 tool calls, 0 parse failures across B1–B3. Cost: lab prefill −8 to −12 % (2,131–2,240). Lab C1 varies 45–56 between 30-s windows on every config (single-sample noise), kit C1 is stable at 47–48. Recommendation for PR #144 / the registry launch on 24 GB cards: --chunked-prefill-size 8192. The agentic 50 tok/s floor is a separate, unchanged result (decision 0011): ~39–40 tok/s at x8.
0013-flashnext-oom-fix.review.md 0013-flashnext-oom-fix.review
AMEND. The two interventions are reasonable, but the plan can declare “fixed” without demonstrating either the cause or §5 qualification. Treat this as mitigation testing; select the fastest qualifying config, with headroom as supporting evidence.
Strongest counter-argument: three clean lifetimes provide weak evidence of reliability. If the true crash probability remains 1/3, three successes still occur with probability 29.6% . With zero failures in three independent trials, the one-sided 95% upper bound on crash probability is 63.2% . Three runs satisfy the spec’s qualification count only when they meet its prescribed conditions; they do not establish that intermittent failure is eliminated.
The diagnosis should remain a hypothesis. Mapping OOMs and exhausted VRAM strongly support memory pressure, but CUDA error: unknown error does not establish the final failure’s cause. A kernel-loading message 29 seconds earlier is correlation. Record the failing allocation/request and available allocator diagnostics; describe the outcome as “memory-pressure mitigation qualified” unless causal evidence improves.
A 0.5-second VRAM sample is insufficient to certify peak headroom. It can miss allocation bursts, and device-free memory alone cannot distinguish reusable allocator reservations from live allocations. Supplement sampling with per-stage allocator peak allocated/reserved memory, allocation-failure records, and request occupancy/chunk size. Keep collection outside the engine and write logs to tmpfs as §9 requires.
Make the comparisons unambiguous. Use baseline (reserve 8.26, chunk 12,288) , A (9.26, 12,288) , and B (8.26, 8,192) . Explicitly reset reserve for B. Test (9.26, 8,192) afterward if necessary, comparing it against either single-change config. This preserves §10’s one-variable-family rule.
Neither adjustment has a guaranteed memory benefit of the stated size. Raising reserve by 1.00 GB may reduce expert-cache capacity; reducing chunk size by 33.3% reduces only chunk-dependent allocations, not total peak usage. Standardize GB/GiB/MiB, record actual cache capacity, and justify the ≥1 GB threshold against observed peaks. It is an engineering margin, not a spec acceptance criterion.
Replace “larger minimum free VRAM and smaller speed cost” with an explicit selection rule. Require lab decode ≥50 tok/s , lab prefill >1000 tok/s , the quality band, and §5 on the same config at the claimed occupancy. Then choose the fastest qualifying config within the search budget. The cited 53.2 tok/s reference permits only about 6.0% decode loss before missing the target; this machine’s measured baseline determines its actual margin.
Make the three finalist runs conform explicitly to §5: cold with all specified caches reset, claimed maximum occupied context, and repeat; zero OOM/crash/parse failures, ≥100 tool calls across the config’s runs, repetition checks, the rolling decode floor, and all six lab gates after each run. A 166k context gate alone does not demonstrate agentic qualification there.
Resolve provenance before running. a21efea6 is insufficient identification, and it differs from the PR #144 digest specified in Phase 1. Record the full digest, source commit, attestation/allowlist eligibility, and reason for replacing the specified image. Store each immutable launch before execution and publish every failed attempt with explicit unavailable metrics.
0014-flashnext-vs-27b.md 0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)0014 — Benchmark Qwen3.8-Flash-Next vs Qwen3.8-27B (operator request, 2026-10-05)
Question: on this box, which is the better local model for agentic coding (and generally): Flash-Next (180B MoE, 6B active, experts streamed from RAM) or the dense 27B the operator runs daily?
Configs (both on GPU2, PCIe 4.0 x8, 15 cores, boost off; one card each)
Flash-Next: PR #144 Phase D launch on a21efea6 with the stability fix (--chunked-prefill-size 8192, decision 0013), EXL3 3.05 bpw, 204,800 context, up to 4 concurrent requests.
27B: the operator's daily server exactly (omarchy-local-ai recipe qwen38-27b-exl3-3bpw-rtx3090-sglang-tp1): ghcr.io/0xsero/sglang-exl3@sha256:84f75f34…, EXL3 3.00 bpw (turboderp/Qwen3.8-27B-exl3@6fe61ad6…), NEXTN/MTP speculative decoding (3 steps, 4 draft tokens), 204,800 context, single stream.
Each is the best known recipe for its model on a 24 GB 3090; they differ in more than one variable by design (this compares models as deployed, not an isolated knob). Sampling, task set, token budget and harness are identical.
Measurements (same order for both; one server lifetime each)
Area Step Metric Speed lab (registry gates), kit sweep, ttftdecode C1 (C2–C4 where supported), prefill 8k–64k, TTFT Long context needle (kit method, thinking off)retrieval at 128k (depths 0.2/0.5/0.8) and ~192k (0.5) Agentic reliability stab (10-min agentic, 8-task §5 set)tool calls, parse failures, loops, decode floor Agentic coding qeval (T07, uncapped, one pass) on tasks-compare30 solve rate (official SWE-bench eval), tokens and wall time per taskFootprint load + VRAM sampler load time, host RAM, VRAM, min free VRAM
tasks-compare30: 30 SWE-bench-Lite test instances drawn with random.Random(20261005).sample (uniform, not hand-picked), images pinned by digest after pull; same 60k completion-token budget per task for both models. With n = 30, a difference in solve rate is reported with a paired comparison (tasks solved by one and not the other) and an exact McNemar p-value; small differences are labelled inconclusive rather than ranked.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Labelled an exploratory deployment comparison on this box (the §10 one-variable rule is waived for it, as the two recipes differ in engine build, MTP and concurrency by design). Conclusions are per priority — agentic coding success, interactive speed, long context, footprint — never "generally better".
compare30 is exploratory: exact McNemar on 30 single attempts needs ≥ 6–0 discordant tasks for p < 0.05 (8–2, a 20-point gap, gives p ≈ 0.11). The site publishes the paired per-task table, the solve-rate difference with a 95 % CI and the p-value; small differences stay unranked.
Cache state: the flow serves cold (container restart, page cache dropped) and then measures in a fixed order, so qeval runs warm for both; the 10-min stab run is screening, not §5 qualification. Flash-Next's kit band is already in band for this config (decision 0013: top-1 0.989, KL 0.00092); the 27B has no kit panel (different model), so its quality evidence is the solve rate only.
Budget: up to 60k completion tokens per task (all of a task's model calls) → worst case 1.8M tokens per model; early submits cut this sharply. Infrastructure failures (image, evaluator) are reported separately from unsolved. Accepted tokens/s and acceptance rate are recorded for the 27B's MTP.
Finding during the 27B run (2026-10-05 18:07 AEST)
The operator's 27B server advertises --context-length 204800, but its KV pool is max_total_num_tokens=135424 (mem-fraction 0.80, fp8 KV, MTP draft buffers). The registry lab's context gate (0.85 × 204,800 = 166,667 tokens) is rejected: "Input length (166667 tokens) exceeds the maximum allowed length (135418 tokens)". So the 27B as deployed really holds ~135k tokens, not 204,800 — the same advertised-vs-real gap seen in the operator's pi session. The comparison reports both; an extra lab run uses the identical server with harness context 131,072 (…-omarchy-ctx131k launch) so its speed gates are measured. Flash-Next's KV pool is 209,984 tokens (fits 204,800).
0014-flashnext-vs-27b.review.md 0014-flashnext-vs-27b.review
AMEND. Comparing the best deployed recipes is useful, and differing engines/MTP settings are legitimate for that question. The strongest objection is inferential: 30 tasks, one attempt each, and one server lifetime cannot reliably establish which model is better “generally.” Frame this as a deployment comparison on this machine, with separate conclusions for coding quality, latency, throughput and resource use.
Thirty tasks can miss substantial differences. With exact two-sided McNemar, 5 wins versus 0 losses gives p = 0.0625 ; 6 versus 0 gives p = 0.03125 . Even 8 versus 2—a 20 percentage-point solve-rate advantage—gives p ≈ 0.1094 . Publish the paired outcome table and a confidence interval for the difference. Keep compare30 exploratory; if a definitive ranking matters, predeclare a larger fixed sample before examining results.
One stab run is screening, not §5 qualification. Unless this exact Flash-Next config already has qualifying evidence, require the three qualification runs: cold, claimed maximum occupied context, and repeat; ≥100 aggregate tool calls; the rolling decode-floor measurement; and post-run lab gates. Otherwise label it experiment-only and its stability unqualified. “One server lifetime” needs an explicit exception for the required cold restart.
The measurement order systematically warms Flash-Next’s streamed experts. lab → kit → ttft → needle → stab → qeval measures coding after extensive preparation, whereas cold agent startup may behave differently. Keep the warm deployment comparison, but add the specified cold qualification and cold first-task latency. Disable prefix caching and use fresh prompts for TTFT; record expert, n-gram and OS cache states separately.
Configured context and needle retrieval do not establish agentic usability. A ~192k needle pass supports retrieval at that occupancy, not reliable coding there. Report server-counted occupied tokens and ≥4k generation headroom. Award “agentic-usable” only after §5 qualification at ≥128k occupied tokens; otherwise say “configured for 204,800.”
Restore the explicit Flash-Next quality gate. Require or cite matching evidence for top-1 ≥0.987 and KL ≤0.0013 against the pinned reference panel before any spec headline or registry claim. SWE-bench solve rate complements that requirement; it does not replace it. Publish outside-band results with the prescribed label.
Make the deployment exception reproducible. Explicitly amend §10’s “one variable family” rule for this comparison and avoid attributing differences to architecture alone. Store full image digests, 40-character weight revisions, file hashes, immutable launches, GPU UUID, harness/evaluator commits, ordered task IDs and the source dataset revision. Record accepted tokens/s, MTP acceptance rate and draft settings as §4 requires.
Account for cost and stochastic variation. The completion allowance totals 3.6 million tokens across both models—20 hours of generation at 50 tok/s , before prefill, tools and evaluation. Reserve that time within the phase budget. Define which tokens count toward 60k, distinguish infrastructure failures from model failures, and report total wall time and tokens per successful solve alongside per-task results. A single attempt measures this run’s outcomes; paired statistics do not eliminate sampling variability.
Concrete alternative: run compare30 as a clearly labelled exploratory deployment comparison; reuse documented qualification evidence where it exactly matches, complete missing qualification/cache checks, and publish a tradeoff table. Recommend a model only for a stated priority—such as coding success or interactive latency. Remove “and generally,” and leave small or uncertain quality differences unranked.
0015-next-tests-plan.md 0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)0015 — Next tests, from the operator's Signal thread with Sero (0xSero) (2026-10-06 … 10-08)
What the thread says
Operator showed local-ai-bench.org/compare and the SWE-bench-Lite contamination finding; asked "Shall I create a benchmark from recent github issues?" — Sero: "Yes" . Operator: "land a novel benchmark, and make it real", and later "an agent/coding benchmark for multiple languages without disclosing the questions/content".
Sero (funding): keep running experiments, turn them into cool demos (Omarchy + the local-ai plugin, "wacky stuff") that he will amplify; fundraiser in a few weeks with 3 goals (small/medium/big) . Hardware advice: stick with NVIDIA (FLOPs, CUDA) — vs the operator's ideas (Epyc 256 GB + 4th 3090, or an M5 Ultra).
Operator wants to go deep on Qwen3.8-Flash-Next ("Qwen 4 is around the corner").
Links: Niko1221/Strata (operator: "amazing decoding numbers … how much better than Qwen 3.8 27B?"); kacper-daftcode/vLLM-Moet + 0xSero/moetier (Sero: "give this to claude your rig will fly", "I don't use gguf"; operator: vLLM-Moet says Blackwell → Sero: moetier "shouldn't need Blackwell"). Operator: "Will test these through the benchmark."
Repo facts: Strata = C++ engine for Flash-Next GGUF quants (Q2_0 … IQ3_S, "Coder" IQ1_M); claims 94 tok/s on a 12 GB RTX 5070 and "100–140 tok/s" expected on a 3090. vLLM-Moet = hand-written SM120 (Blackwell) SASS kernels — cannot run on SM86 3090s. moetier = Sero's data-first standard + simulator for expert placement across VRAM/RAM/NVMe; its loop's primary target is GLM-5.3-Flash on 1× 3090 + 55 GB DDR4 + NVMe at 50 tok/s , with run records (registry/runs) carrying a standard table, utilization and chokepoints (moetier probe).
Plan (ordered; each item ends in published records)
Tell Sero the PR #144 stability finding now (operator sends; draft in prs/pr144-proof.md): a21efea6 runs a 24 GB card to 2 MiB free and crashed 2/4; --chunked-prefill-size 8192 fixes it (3/3, decode unchanged). He is the maintainer, it is the cheapest high-value result we have.
moetier on this box (Sero's explicit ask). (a) hardware record for omarchy-gpu (GPU2 x8 13.5 GB/s, 2-ch DDR4, 16.6 GB/s memcpy, 1 TB NVMe 3.4 GB/s rand) and a Flash-Next model record if missing; (b) moetier probe on the trellis Flash-Next config → utilization + chokepoints run record: name the binding constraint behind the ~40 tok/s agentic floor at x8; (c) reproduce his GLM-5.3 NVMe-mode recipe at a 55 GB RAM budget on our single NVMe (an independent hardware datapoint for his G1/G2 work); (d) moetier sim the fundraiser hardware goals.
Strata vs 27B vs trellis Flash-Next (operator's question). Pinned Strata release; Flash-Next GGUF quants (Q2_0, IQ2_XS, IQ3_S, Coder) on GPU2, then 2-3 cards GPU2-first. Same suite as decision 0014 (lab, kit sweep, TTFT, needle, 10-min agentic) + quality vs the EXL3 3.05 reference (KLD/top-1 where the API exposes logprobs). Test the "100–140 tok/s on a 3090" claim. GGUF → operator-facing comparison, not a registry recipe (Sero).
Fresh-issues benchmark (decontaminated, held out). Pilot: ~50 Python tasks from issue-linked PRs merged after 2026-09-01 (after both models' release), auto-built Docker envs (reuse SWE-rebench / SWE-bench-Live RepoLaunch tooling), validated FAIL_TO_PASS/PASS_TO_PASS with the gold patch, run through our harness. Held out: publish aggregate scores, repo list, task-ID hashes and a few retired samples; refresh monthly. Then add TypeScript, Go, Rust (multi-language, per the operator). Keep the gold-overlap check as a standing guard.
Run the three contenders on it : 27B, trellis Flash-Next (fixed), best Strata quant; paired analysis.
Demos for Sero / the fundraiser : Omarchy + local-ai plugin — (a) 27B as a fast coding agent (one-shot app), (b) Flash-Next answering over a ~190k-token codebase; short screen recordings + numbers from the records.
Fundraiser evidence : per goal (small: 4th 3090 / 2nd NVMe; medium: Epyc 8-ch 256 GB + x16 lanes + 4th 3090; big: a Blackwell card → vLLM-Moet/moetier), projected tok/s from moetier sim calibrated on our measured runs.
Qwen 4 day-0 readiness : launch templates + the fresh-issues set ready so a new model runs the same day.
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Milestone 0 first: Phases 1–2 are closed as documented results (0011, 0012); their Phase 3 material (PR #144 evidence, kit submission) goes out before new work is presented as findings. PR #144 wording: "no crashes in 3 full-sequence runs with chunked prefill 8192 (vs 2 of 4 without); decode unchanged within ±2 % (kit C1 47–48); prefill −8 to −12 %" — a mitigation, not a proven fix.
moetier: answer one measurable question — what binds the ~40 tok/s agentic floor (bytes per token over PCIe, expert-cache misses, CPU lane, GPU idle) — with measured counters, then change one variable family. The GLM reproduction states its RAM budget in GiB including pinned memory, swap off, cache resets declared; 50 tok/s is Sero's hypothesis, and registry gates still apply.
Strata: screen quants on GPU2 only (one 10-min agentic run + speed suite each), qualify only finalists, and add 2–3 cards only for a finalist with a measured reason. Verify what "Coder" changes (weights/behaviour). Quality without logprobs = "unavailable", never in-band.
Fresh-issues set = "temporally held out" , not "decontaminated": require issue and fix after 2026-09-01, overlap check against existing benchmarks, no network access for the agent during evaluation, tasks frozen before any tuning. A separate small development set is used for choosing configs; finalists run once on the held-out set. 50 tasks is a pilot (±14 pp at 50 %); report paired, repository-aware uncertainty.
Secrecy vs reproducibility: a private immutable manifest (commits, patches, tests, env digests, harness settings) with a published commitment hash; controlled reproduction on request; full artifacts released when a task set retires. Active-set raw logs stay out of the public site (SPEC §8 amended accordingly).
Fundraiser numbers are conditional projections with stated assumptions and sensitivity ranges, kept apart from measured results; topology (slots, lanes) is checked before any 4th-3090 projection.
0015-next-tests-plan.review.md 0015-next-tests-plan.review
AMEND. The strongest objection is scope drift: the agreed deliverable is qualified Flash-Next and single-3090 GLM recipes; this plan adds a benchmark product, several engine comparisons, demos and hardware projections before closing those deliverables. Keep the new work, but put it behind an explicit milestone: publish the fastest qualifying configs—or documented failures within the spec’s timebox—and prepare their submissions.
Report PR #144 as a promising mitigation, not a proven fix. Moving from 2/4 crashes to 0/3 is useful evidence, but three successes still permit a per-run failure probability of 63% at a one-sided 95% confidence bound under independent, identical trials. Use “no crashes in three qualification runs,” verify the same occupied context and workloads, and quantify “decode unchanged.” Confirm Phase 3 publication requirements are already satisfied or explicitly amend them.
Make moetier answer a measurable bottleneck question. Going from ~40 to 50 tok/s requires 25% more throughput , or 20% less time per token. If PCIe alone were saturated at 13.5 GB/s, that implies reducing traffic from approximately 338 to 270 MB/token . These are conditional bounds, not a diagnosis. Measure transferred bytes, expert-cache misses, CPU work and GPU idle time, then change one variable family. A utilization report alone cannot establish causality.
Define the GLM reproduction before running it. Specify whether “55 GB” means decimal GB or GiB, and whether it includes pinned allocations, page cache and engine overhead; declare swap and cache-reset behavior. Treat 50 tok/s as the supplied recipe’s hypothesis, not an expectation. Registry eligibility still requires the ≥15 tok/s floor, uncapped quality evaluation and ≥80% of reference solves; reproducing a fast simulator or engine run does not satisfy those gates.
Separate Strata’s speed claim from its eligibility. The supplied 100–140 tok/s claim is 2–2.8× the Flash-Next target, but needs matching context, cache state, sampling and accepted-token accounting. A low-bit quant can win throughput and fail quality. Missing logprobs mean “quality unavailable,” which cannot establish the required top-1 ≥0.987 / KL ≤0.0013 band. Verify whether “Coder” changes weights or model behavior before presenting it as a quant-only comparison.
Rename the fresh set “temporally held out”; “decontaminated” overclaims. A PR merged after model release can fix an issue published long before release, and an agent with network access can retrieve the public solution. Require fresh issue/test provenance, inspect overlap with existing benchmarks, block solution retrieval during evaluation, and freeze evaluation tasks before tuning. These controls reduce contamination risk; they do not prove its absence.
Treat 50 tasks as a pilot, not a decisive leaderboard. At a 50% solve rate, 50 independent tasks give an approximate 95% margin of ±14 percentage points ; one task changes the score by 2 points. Repository clustering can make uncertainty larger. Report paired outcomes and uncertainty based on discordant tasks, with repository-aware analysis. Use a development set for choosing the “best Strata quant,” then evaluate finalists once on the held-out set.
Resolve secrecy versus reproducibility explicitly. Public aggregate scores, repository names and task-ID hashes do not let another evaluator reproduce scores, and hashes of enumerable public IDs may reveal tasks. Keep a private, immutable manifest containing base commits, patches, tests, environments and harness settings; publish its commitment and arrange controlled independent reproduction. Amend §8 so public raw logs cannot accidentally expose active tasks, with full artifacts released upon retirement.
Make fundraiser projections conditional, with uncertainty. A fourth 3090 adds 24 GB nominal VRAM , not guaranteed usable bandwidth or speed; the actual slot, lane and chipset topology must be established first. A second NVMe helps only if measured storage contention is binding. EPYC and Blackwell projections require calibration beyond this AM4/SM86 machine. Publish assumptions and sensitivity ranges; keep measured results separate from simulated forecasts.
Concrete replacement order: narrow PR #144 report → close existing Flash-Next/GLM qualification and submission material → probe the measured bottleneck → screen Strata on GPU2 and qualify only finalists → freeze and validate the 50-task pilot → paired contender evaluation → demos and conditional fundraiser scenarios. Four quants across three layouts already imply 12 configs: up to eight hours of screening plus qualification alone , before loading, lab, sweep and quality checks. Reserve multi-card expansion for a single-card finalist with a demonstrated reason to benefit; label the ~190k demo by actual occupancy and qualification status.
0016-fresh-issues-stage2.md 0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")0016 — Fresh-issues benchmark, stage 2: environments, validation, freeze (pilot "fresh-2026-09")
Context: decision 0015 item 3 (temporally held-out Python set, issue AND PR created ≥ 2026-09-01, private tasks, published scores + commitment hash). Stage 1 (tools/fresh/, data in ~/dev/local-ai-exp/fresh-private/, never in this repo) gave a pool of 5,450 candidates and a ≤ 3-per-repo shortlist of 763 (275 repos; 249 easy / 271 medium / 243 hard; 528 from repos not in any public SWE dataset). Operator choice (2026-10-08): the setup/filter LLM is this Claude subscription (claude -p, Opus) — not an API key; Anthropic therefore sees which tasks we chose ("semi-private for Anthropic models", stated next to every score).
Plan
Stage-2 input (deterministic, before any LLM): from the shortlist drop rows with base commit dated before 2026-08-24 (5), no .py test file (5), and repos created after 2025-09-01 or with < 10 distinct contributors in the last year (guards against single-author agent-generated issue→PR streams seen in the top repos). Sample 150 by difficulty (easy 45 % / medium 40 % / hard 15 %), seen/unseen repos ≈ 1:2. Seed fixed, list recorded.
Clarity filter (Opus, one claude -p call per task, JSON out): the SWE-bench-Verified rubric — is the issue well specified, can a correct fix be judged by the hidden tests without guessing names/messages that only the tests define, is the fix given away in the issue. Reject scores ≥ 2 (0–3 scale) on either axis.
Environment (deterministic skeleton + agent-written setup):
Repo snapshot: git init + git fetch --no-tags origin <base_commit> (ancestors only; no branches, tags or later objects exist in the image), checkout detached, remote removed, reflog expired, gc --prune=now; a post-build check asserts git rev-list --all contains no commit dated after the base and the gold blobs are absent.
Base image python:<ver>-bookworm pinned by digest. Opus (claude -p --model opus, Bash tool restricted to docker exec into a throwaway per-task container with network, 45-min cap, ≤ 60 turns) writes setup.sh (system packages, pip/uv installs) and the test command pytest -rA <test files>; installs pinned to versions available before the PR's creation date where the agent can (uv pip --exclude-newer). The final image is rebuilt from setup.sh without the agent (reproducible), tagged lab/fresh.<id>, digest recorded.
LLM-written log parsers are not used: pytest -rA + swebench parse_log_pytest only (no host-side code from the agent).
Validation (no LLM): in the image, run the test files on base + test_patch (expect failures), then base + test_patch + gold patch; FAIL_TO_PASS = fail→pass, PASS_TO_PASS = pass→pass. Require ≥ 1 F2P, gold resolved 3/3 (flaky tests dropped from P2P, task dropped if an F2P is flaky), empty patch resolved 0/1, test run < 10 min. Rows emitted in our swebench 5.x format (eval_script, parse_log_pytest, version fresh-2026-09, image digest) into a private task dir; swebench evaluates them with network off for the agent under test.
Freeze: from validated tasks, re-apply ≤ 3 per repo, stratify, pick 50 test + 10 dev (dev is for harness debugging only; never reported). Manifest (ids, base, patches, digests) committed only in fresh-private; publish sha256(manifest || salt) and the counts/distribution now, the salt and manifest when the slice is retired.
Runs: Qwen3.8-27B (ctx131k), Flash-Next trellis c8192, Strata finalist; same harness as compare30 (uncapped, one pass), paired analysis (McNemar), gold-overlap per task as a leak check (flag > 50 % verbatim reproduction).
Scheduling and cost
Stage 2 is CPU/disk heavy (docker builds, test runs): it runs only as sh lines of the campaign queue, never during a GPU measurement. Parallelism 3 tasks. Disk: ~1–2 GB per image → ≤ 200 GB worst case before pruning failed builds. Subscription usage: ~150 filter calls + ≤ 150 setup sessions; stop and report if rate-limited.
Risks
Yield may land < 50 (published Python validation yield ~19 %): widen to the pool (reserves) and to October issues as a second slice rather than relaxing the date rule. A repo snapshot without later commits can still contain the fix if it was committed to the base before the PR (rare; the gold-overlap check at run time flags it).
Outcome after review (GPT-6.1 Sol: AMEND) — accepted
Supplementary pilot , contamination-reduced (not "uncontaminated"); labels next to every score: "Opus-curated", "semi-private for Anthropic models", rubric + Opus id + rejection counts published. Does not replace §5 qualification or the reference panels. Public release only after Phase 3 review.
Ordered batches, not 150: an ordered September list (stage-2 filters applied, stratified, seed fixed) is frozen now; attempts run in batches of 50 until 60 validated (expect ~320–380 attempts). Budgets: setup cap 30 min/task, ≤ 3 parallel, stage-2 wall time ≤ 40 h, image disk ≤ 150 GB (rejected images pruned immediately). If a budget runs out, stop with an undersized pilot and say so. October = separate slice.
Git invariant: image HEAD == base SHA; refs = HEAD only; git rev-list --all == ancestry of base; no stray working-tree files vs git status; gold-patch files are never mounted or copied into setup containers. Base must fail the hidden F2P tests (acceptance check).
Reproducibility: keep setup.sh, resolved pip freeze/uv lock with hashes, apt package versions, build logs, test command and image digest. Setup containers: CPU/RAM/PID limits, no host secrets, no Docker socket.
Validation: collected node IDs must match between runs; skips/xfails/collection errors counted explicitly; test patch must not disable tests; each repetition in a fresh container; gold 3/3 + empty 0 recorded with every exclusion and reason (3/3 is weak evidence on flakiness — reported as such).
Protocol frozen before candidate runs: manifest bytes, high-entropy salt, ordered reserves, selection rules, harness/parser SHAs, sampling, per-task token budget (same for all models; the compare30 harness settings), tool budget, context policy, retries, timeout treatment. Weight revisions recorded per model.
Reporting: one primary comparison predeclared — Strata finalist vs Qwen3.8-27B (the operator's question); others exploratory. Discordant counts, exact McNemar, repo-clustered intervals, ±14 pp margin stated at n = 50.
Gold overlap: descriptive audit only (denominator = gold added lines), never used to exclude or rescore.
Predeclared before any candidate run (2026-10-09 05:50 AEST)
Strata finalist = IQ3_S (launch strata-qwen3.8-flash-next-gsq-rco-iq3_s-128k-rtx-3090-24gb): T41 screen agentic floor 87.3 tok/s (Q2_0 123.3), compare30 29/30 (Q2_0 28/30); chosen as the higher-quality quant that still clears the 50 tok/s floor. Primary comparison: Strata IQ3_S vs Qwen3.8-27B (ctx131k launch) , exact McNemar on the 50-task test slice. Exploratory: Flash-Next trellis c8192, Strata Q2_0.
Harness: tools/agentic/quality_eval.py at the commit recorded in each result, uncapped one pass, 60,000 completion tokens per task, sampling temperature 0.7 / top_p 0.95 / seed 1234 / max_tokens 8192 (compare30 settings), agent container network none.
0017-site-structure.md 0017 — Site structure: story-led home, findings, recipes, model cards0017 — Site structure: story-led home, findings, recipes, model cards
Operator request (2026-10-10): pages were messy; home must show the sequence of what was done and a summary of achievements; every model gets a model card (link, details, learnings to run). Debated with GPT-6 Astra (Codex, 2 rounds) before building.
Outcome
Navigation: Home · Findings (speed, coding quality, GLM-5.3-Flash, hardware) · Recipes · Models · Method & Hardware · Runs & Data · Funding & Contributions. Old tabs (#charts, #compare, #fresh, leaderboards) kept for links, marked superseded or moved under Data.
Home: dated current answer (speed config and task-success config named separately), three bounded achievement summaries with conditions/limits/evidence, the 10-step dated timeline (outcome-led, chips, decision links), contributions with real status, hardware hypotheses, next steps.
Editorial text lives in site/content.json (operator-reviewed); charts and numbers in tables are computed from records.
Proposal (Claude)
local-ai-bench.org — information architecture proposal (draft by Claude)
Context
Static site generated from ~250 result records (tools/site/build.py). Audience, in order: (1) 0xSero, maintainer of the local-ai registry/recipe-kit and our lead, (2) potential fundraiser backers (hardware goals small/medium/big), (3) local-LLM practitioners who want a recipe for 3090-class boxes. The operator says the current pages "look very messy between them".
Current tabs (10 + 247 run pages)
Overview (per-model status cards, PR list, recent runs, a new headline callout) · Flash-Next leaderboard · GLM leaderboard · 27B leaderboard · Fresh-issues benchmark (method + results mixed) · Flash-Next vs 27B (public SWE-bench-Lite comparison, memorised) · Charts (8 charts with no narrative) · Runs (247) · Hardware & methodology (hardware, rules, decisions, spec). Problems: the story is nowhere; results for one question are spread over 4 tabs; three near-identical leaderboards; charts detached from the claims they support; method and results mixed; the old compare page contradicts the new benchmark unless read with a caveat.
What was done (chronological, all dated AEST, each backed by records + a decision file)
Oct 3–4 · Box baseline: 3x RTX 3090 on AM4 (GPU2 x8 13.5 GB/s, GPU0 behind chipset 6 GB/s), faulty CPU core found and quarantined, boost off after machine-check errors (decisions 0004, 0010). Card order matters: GPU2-first +43–51 % prefill (0006).
Oct 4 · Flash-Next on trellis/SGLang (Sero's PR #144 launch): lab 55 tok/s, prefill 2,400, but agentic floor ~40 at x8 — 50 not reached in any tested x8 config (0011). x16 card move dropped by the operator (0009).
Oct 4–5 · GLM-5.3-Flash on one 3090: every route < 15 tok/s (0012).
Oct 5 · PR #144 image OOM found (2/4 crashes) and mitigated with chunked prefill 8192 (0013). Public site launched.
Oct 5–6 · Flash-Next vs 27B on public SWE-bench-Lite: 30/30 vs 18/30 — but Flash-Next reproduces upstream fixes verbatim (memorised) (0014).
Oct 8 · From the Sero thread (0015): PR #144 rerun comment + kit submission PR posted; Strata (C++ GGUF engine) screened: Q2_0 floor 123 tok/s, IQ3_S 87 tok/s on one x8 card; moetier on the box: GLM NVMe recipe 6.1 tok/s at 55 GiB → 11.3 at 90 GiB; probe shows NVMe-bound → CPU-bound; trellis Flash-Next is PCIe-bound.
Oct 8–10 · Built a private fresh-issues benchmark (0016): 430 attempts → 50 tasks, issue+PR after 1 Sep 2026, commitment hash published. Strata IQ3_S 48/50 vs 27B 36/50 (exact McNemar p = 0.0018, predeclared).
Proposed structure (5 top-level sections)
Home = the story. Hero: the current answer in one sentence + 3–4 achievement tiles (fastest agentic setup on one 3090; best coding quality on fresh tasks; GLM verdict; contributions upstream). Then the dated timeline above as a vertical sequence, each step with its one key number, a status chip (result / negative result / fix / tool) and a link to the finding. Ends with "What's next" (Qwen 4 readiness, multi-language slice, fundraiser hardware sims).
Findings (one page per question, narrative + the charts that support it + links to runs): a. Flash-Next speed on one RTX 3090 (engines compared; why x8 PCIe caps trellis; Strata result; stability fix) b. Coding quality (fresh-issues results; the memorisation finding on public tasks; 27B vs Flash-Next) c. GLM-5.3 on one RTX 3090 (routes tried; moetier NVMe recipe; what limits it; what hardware would fix it) d. Hardware lessons (PCIe lanes and card order, faulty core, NVMe at x2, RAM caps)
Methods : how we measure (lab gates, kit sweep, TTFT, stability floor, quality panel, probe) and the fresh-issues benchmark construction (moved out of the results page), hardware record, rules for statuses.
Data : one filterable config table (model/engine/quant/cards filters, replaces the three leaderboards), all runs, run detail pages, decisions log.
Upstream : PRs and submissions to Sero's repos with status, and recipes ready to copy (launch commands).
Charts tab removed: every chart moves into the finding it supports. The old "Flash-Next vs 27B" page becomes a section of Findings b, framed as the memorisation evidence.
Questions for the reviewer
Is this the right structure for these audiences? What would you cut, merge or reorder? Is a timeline the right home page, or should the home lead with findings? Anything that makes the site look less credible?
Round 1 (GPT-6 Astra)
The proposal fixes the fragmentation, but risks replacing ten messy tabs with a polished lab diary. Lead with findings and usable recipes; make the chronology secondary. A public visitor needs to know what works, under which conditions, and how strong the evidence is.
0xSero’s priorities deserve a prominent contribution summary, not control of the entire navigation. Practitioners need reproducible setups; backers need evidence that a specific purchase would resolve a measured constraint. Neither audience benefits from reading seven dated steps first.
I would use five sections, in this order:
Results — question-led findings, with a comparison summary.
Recipes — tested launch commands, requirements, expected performance, limitations.
Method & Hardware — measurement definitions, benchmark construction, exact machine configurations.
Runs & Data — filterable results, individual runs, downloadable records, decisions.
Project & Funding — upstream contributions, dated progress log, hardware proposals.
Home remains accessible through the logo. “Upstream” is insider language and combines two different jobs: tracking contributions and getting software running. Recipes should not be buried there.
The home page should contain, in order:
A bounded answer: which tested configuration you currently recommend for which workload, on what hardware, as of what date.
Three finding cards: practical agent performance, fresh-task coding results, and GLM feasibility. Each needs a configuration, metric, limitation, and evidence link.
“Run a tested setup” linking directly to recipes.
“What additional hardware would test” showing small/medium/big funding options, the measured constraint each addresses, cost, and success criterion.
“Recent changes” with three entries and a link to the full timeline.
Cut the achievement tiles. “Fastest agentic setup on one 3090” is especially risky when the supporting numbers mix laboratory throughput, agentic floors, engines, and quantizations. “Best coding quality” overstates a two-configuration, 50-task comparison. Use “Highest measured [defined metric] among tested configurations” and “48/50 versus 36/50 on this task set.” Put contributions below results, not alongside them as equivalent scientific achievements.
Keep the question-led findings, but sharpen their titles:
Which setup runs coding agents fastest?
How often do the tested setups solve fresh issues?
Is GLM practical on this machine?
Which hardware constraints affect performance?
The largest credibility issue is comparability , not navigation. “One RTX 3090” obscures a three-card host with unusual lane allocation, CPU restrictions, RAM requirements, and NVMe constraints. Display active GPUs separately from host hardware. Every comparison should expose model, engine, quantization, workload, context length, and relevant resource settings.
The fresh benchmark needs a compact evidence box beside its headline—not only a distant Methods link. Explain how 430 attempts yielded 50 tasks, exclusions and their timing, whether selection involved the evaluated systems, paired outcomes, and what was predeclared. A commitment hash needs a timestamp and a clear account of what it commits to. Recent issue dates alone do not establish contamination freedom; the p-value does not establish general coding superiority.
Keep chart integration, consolidated data, negative results, and the decisions log. Preserve old URLs with conspicuous superseded-result notices. Describe verbatim fixes as evidence consistent with memorisation unless stronger provenance establishes it. Treat proposed hardware remedies as hypotheses until tested.
Round 2 (GPT-6 Astra, after the owner's constraint)
The owner’s constraint changes my recommendation: make chronology the home page’s main structure, with the current answer and achievement summary above it. I withdraw the blanket objection to tiles; bounded result summaries serve a useful purpose.
1. Home, top to bottom
Current answer. A short, dated statement distinguishing the fastest tested configuration from the configuration evaluated for task success. Explicitly separate Q2_0’s speed result from IQ3_S’s 48/50 result. Name the workload, active GPU count, and three-card host; link to the applicable recipes.
Three achievement summaries: agentic performance, fresh-task outcomes, and GLM feasibility. Each gets a metric, essential conditions, limitation, and evidence link. Keep them compact enough that the answer and summaries fit a typical desktop first screen; on mobile, prioritize the answer. Put the fresh-benchmark evidence box directly beside or beneath its summary, not behind a tooltip.
“How we reached this answer.” The dated, chronological sequence is the page’s main body: 6–8 outcome-led steps, each linking to its finding and underlying records. Show the initial constraint, subsequent result, and any change in interpretation. Include the public-benchmark result together with the memorisation caveat.
Contribution summary. Show the stability work, rerun report, and kit submission with accurate statuses and links.
“Run a tested setup.” Direct recipe links with requirements and expected results.
“What additional hardware would test.” Measured constraints, proposed purchases, costs, hypotheses, and success criteria.
2. Final top-level navigation
Home → Findings → Recipes → Method & Hardware → Runs & Data → Funding & Contributions
Home deserves an explicit label because the owner makes the sequence a primary destination. Combining funding and contributions keeps project support together without burying recipes.
3. Remaining critical objections
Your timeline revision works, but “one number per step” should be a preference, not a rule . Some outcomes require a denominator, before/after values, or paired conditions. Clarity matters more than visual uniformity.
The proposed tiles still need corrections:
“Strata Q2_0” identifies an engine and quantization, not the model. Include the model and define “floor”; ten minutes alone does not explain that metric.
“GLM-5.3” must preserve the exact tested variant, apparently GLM-5.3-Flash , and say “across tested routes on this host.”
“Predeclared” must identify what was declared and when. The p-value belongs with paired outcomes and selection disclosures.
“2 upstream contributions” must distinguish submitted, accepted, and merged work.
Most critically, never combine Q2_0 speed and IQ3_S task success into an implied single configuration’s achievement.
Budget and stop-rule log (budget.md) Budget
Phase Started Budget Status 0 setup 2026-10-02 1 day in progress 1 Flash-Next – 2–3 days – 2 GLM – 2–3 days – 3 review & PRs – operator-paced –
Stop-rule log
Date Route Last 3 gains Decision
Site rules Status badge not started = no records; done = a config with a kit / registry / kit+registry / experiment-only eligibility whose stability verdict is qualified; otherwise running. site/status.json overrides it (shown as override). Stability verdict failed = any stability run failed or crashed, or shows parse failures, repetition hits or crash/OOM; qualified = at least 3 passing runs, one of them cold, and at least 100 tool calls across them; screened = passing runs short of that. Flash-Next target one passing lab record of the config with decode >= 50 tok/s and lab prefill > 1000 tok/s. Headline = the qualified target-meeting config with the largest occupied context reached in its passing stability runs. GLM target best lab C1 decode of a 1-card config against the registry speed gate (pins.json). 3-card configs are shown separately and do not satisfy the 1x RTX 3090 request. Agentic-usable qualified and a passing stability run reached >= 128k occupied tokens; otherwise the label is 'configured for N k'. Kit band top-1 >= 0.987 and mean KL <= 0.0013 (Flash-Next, quality_panel records). GLM quality floor solve-rate ratio >= 80% of the reference (metric solve_rate_ratio on quality_eval records, else computed from the model's eligibility=reference quality_eval). Dropped config a config with a failed or crashed record and no record carrying a positive eligibility.
Pins 0xSero/local-ai-registry d21258dd744e7c78be28177c90060c6af8e10b7e0xSero/local-ai-recipe-kit ef883d269e50ecbea290f919f095e2f3ca633b42SWE-agent/mini-swe-agent {"commit": "04d809ceab9df28f9adaed044884180159172930", "nearest_tag": "v2.4.6"}panel qwen3.8-flash-next-exl3 {"path": "reference/qwen3.8-flash-next-exl3-ref-panel.json", "sha256": "3350eef4e980779256a0c1e872d9345d0d21bdf43690e1958526aef17e02488d"}panel glm-5.3-flash-exl3 {"source": "0xSero/local-ai-recipe-kit PR #3 @ b5d0d3c2d567c4d4c1a295acc0a0fc86470b9097", "path": "artifacts/glm-5.3-flash-exl3-ref-panel.json", "sha256": "3445baefa35851f7f431b4f46fc2266a9d234d79526b1d4fc1216093f552f269"}lab gates load, chat, reasoning, tools, context, speed lab speed_min_tps 15.0 lab speed C1 streamed decode over first 30 s after first token, temperature 0.8, uncapped answer lab context needle in a prompt of ~0.85*ctx; pass if recalled and prompt_tokens >= 0.6*ctx lab chat finish_reason=stop with non-empty content lab reasoning separate reasoning_content and 17*23=391 in content lab tools get_weather(city=Paris) call, then tool result (17) used in reply lab prefill_definition context-gate prompt_tokens / total request seconds (includes generation) kit early_exit_threshold 0.95 kit early_exit_policy disabled during exploration (spec §4)
pins.json sha256 858564b6e07125ac48c2ef6c5178d0a53596f21eb5f0ebda43906472b01b4d78, pinned 2026-10-02
Protocol (from SPEC.md) 2. Hardware ("the ryzen", dedicated to this work)
Part Detail CPU Ryzen 9 5950X, 16C/32T, AVX2, no AVX-512 Board ASUS Pro WS X570-ACE RAM 128 GB DDR4-3200, dual-channel (4×32), ~125 GiB MemTotal GPU 3× RTX 3090 24 GB, driver 610.57.04 PCIe today GPU0 04:00.0 x8 behind the X570 chipset; GPU1 0b:00.0 x8 CPU (drives display); GPU2 0c:00.0 x8 CPU PCIe x16 layout one card alone in the primary CPU slot (operator moves cards) Storage 1 TB NVMe, ~748 GB free: active models. 20 TB ZFS HDD: unused
Identify GPUs by UUID in every result. Never benchmark on the display GPU. 3-card runs are headless : the operator logs out of Hyprland and everything is driven over SSH; each record notes display: off.
4. Definitions and metrics
Official speed : the numbers from lab/lab.py try --on endpoint in the registry. Decode = C1 over the first 30 s at T=0.8. Lab prefill = context-gate prompt tokens ÷ total request seconds.
Lab prefill is a conservative number. A value > 1000 counts as proof that prefill clears 1000. Record time-to-first-token prefill separately, with the prefix cache disabled and fresh prompts.
Pinned harness : at Phase 0 start, record the commit SHAs of the registry, kit, mini-swe-agent and the reference panels. Every run cites them, and the six lab gates and thresholds are those at that commit.
Decode floor measurement : generation-only tok/s (output tokens ÷ time spent streaming them, excluding prefill, tool execution and idle time) over rolling 60 s windows, ignoring the first 60 s. "Holds the floor" = every window ≥ the floor. This is separate from official lab C1 and from the whole-run average.
Supplementary speed : the kit PROTOCOL.md sweep (prefill 8k/16k/32k/64k, decode C1–C4), always run with early exit disabled while exploring.
Real-world speed : decode measured across the whole agentic run.
Context is reported as occupied tokens (the prompt actually sent), alongside the configured limit. Maximum tested: 260k. The label "agentic-usable" is awarded only after a config passes §5 qualification at ≥ 128k occupied tokens ; otherwise it shows "configured for N k". Shorter profiles are still published.
Occupied context = prompt tokens counted by the served model's own tokenizer (usage.prompt_tokens from the server), excluding generated tokens. A max-context claim needs evidence that a request actually reached that occupancy, with ≥ 4k tokens of generation headroom left.
MTP / speculative decoding : always report accepted tokens/s, acceptance rate and the draft settings.
5. Stability and quality protocol
Run types
Stability runs : capped at 10 min each (screening and qualification below).
Quality evaluation (GLM solve-rate, §5 GLM quality floor): a separate run with no 10-min cap but a fixed token budget per task. Finalists only.
Agentic harness
mini-swe-agent, pinned commit.
Fixed set of 5–10 SWE-bench-Lite tasks, at least one growing past 64k context.
Sampling is fixed: temperature, top-p, seed where supported, max tokens.
Screening run
Finalist qualification
3 runs of ≤ 10 min each : one starting cold, one at the claimed maximum occupied context, one repeat.
Cold = container restarted (prefix/KV and expert caches empty), the n-gram row cache empty, and the OS page cache dropped (sync; echo 3 > /proc/sys/vm/drop_caches). Each record lists which caches were reset.
Each run works through the task set in fixed order, looping, until the 10 min wall clock runs out. An unfinished task counts as unsolved.
Pass criteria (every run)
No crash and no OOM.
0 tool-call parse failures . The ≥ 100 tool-call minimum is counted across all runs of a config (screening + qualification + extra 10-min runs as needed), not per run.
No repetition loops. Detector (frozen per decision 0003): any 64-token sequence repeated ≥ 3 times within one field of a response (reasoning, content, or one tool call's arguments, each checked independently; cross-field matches are logged as diagnostics only), or the same tool call (name + arguments) issued ≥ 4 times in a row within one agent conversation (the streak resets only when a fresh conversation starts). When it triggers, save the offending response. Tokens come from the served model's tokenizer when the server exposes one, else tiktoken cl100k (recorded).
Error categories are reported separately : malformed tool calls (invalid argument JSON, unparsed tool-call markup, unknown tool, missing command) are tool-call parse failures ; responses cut off at max_tokens without a tool call are output truncations ; other agent-protocol errors are agent format errors . Only the first category is the "0 parse failures" criterion; the other two are published next to it.
Decode stays at or above the absolute floor for the whole run: Flash-Next ≥ 50 tok/s; GLM ≥ 15 tok/s when the config is a registry candidate.
All six registry lab gates still pass after the run.
GLM quality floor
Passes the registry tools gate.
Meets the parse-failure rule above.
Solves ≥ 80 % as many tasks as the reference on the same task set, measured in the uncapped quality evaluation with identical task IDs, sampling settings and per-task token budget for candidate and reference. If the reference solves 0 tasks, the solve-rate criterion is reported as not measurable and the config cannot be a registry candidate on quality grounds.
Reference: GLM exact mode (bit-exact EXL3 3.05 bpw). If exact mode cannot load, the reference is the highest-bit GGUF that fits.
Also report KLD and top-1 agreement where they are available.
Flash-Next quality
Kit band: top-1 ≥ 0.987 and KL ≤ 0.0013 against the exllamav3 reference panel (tools/score_ref_panel.py, the kit's pinned panel).
The band is required for the headline result and for the registry PR . Configs outside it are still submitted to the kit and published, labelled "outside band" (precedent: kit PR #1).
Generated 2026-10-12 18:13 AEST by tools/site/build.py from 281 records in results.