Server 120 — модели и фактические тесты
Ryzen AI Max+ 395 · Radeon 8060S / Strix Halo · gfx1151 · 128 GB UMA, рабочее окно около 96 GiB · Ubuntu 24.04 · обновлено 05.09.2026
Последний production baseline: Qwen3.6-35B-A3B Q4_K_M; сервис остановлен на время acceptance-серии. Все остальные модели тестировались изолированно. Этот каталог показывает не рекламные цифры, а локальные измерения и ограничения.
Быстрый выбор
| Задача | Модель | Почему | Ограничение |
|---|---|---|---|
| Общий production text/agent | Qwen3.6-35B-A3B Q4_K_M | Лучший подтверждённый баланс RU/EN, tools и скорости | Следовать production profile; не менять sampler без A/B |
| Sandbox tool/repo | Qwen-AgentWorld Q4_K_M | 50/50 tool loop, recovery, repo-edit, security PASS | Только безопасный sandbox |
| Fast helper/router | Granite 4.2-8B / LFM2.5 | 38–39 tok/s / 211–221 JSON tok/s | Verifier обязателен; не primary model |
| Vision staging | Nemotron Omni Q4 | 10/10 image transport, 60.18 decode tok/s | Grounding 1/4, video abort |
| Research long context | Flash-Next MTP | Работа до 245K, MTP +18–20% на коротком A/B | Coding/strict output fail; production reject |
Проверенные конфигурации
Общие правила запуска
HSA_USE_SVM=0 HSA_ENABLE_SDMA=0 llama-server -m MODEL.gguf --host 127.0.0.1 --port PORT \ --ctx-size CONTEXT --parallel 1 --gpu-layers 999 \ --jinja --flash-attn auto --seed 42
- Тяжёлые модели запускать последовательно, не одновременно на UMA.
- Для production Qwen3.6 закреплены
--reasoning off --flash-attn off --no-mmap. - Для tools: строковый
tool_choice:"required",enable_thinking:false, достаточныйmax_tokens. - MTP с
parallel>1считать недопустимым до отдельного slot-isolation gate. - Фактический context считать по
/tokenizeиusage.prompt_tokens.
Основные модели
Qwen3.6-35B-A3B Q4_K_M — 20.42 GB — PRODUCTION BASELINE
Runtime: b10276 ROCm/HIP gfx1151, -ngl 999, parallel=1; production: reasoning off, flash-attn off, no-mmap.
Speed: 45–60 tok/s short decode; control 58.83–59.54. Tools: corrected 15/15 short/8K/30K. Business: selection 10/10, exact args 9/10; date retry 5/5.
Recommendation: retain as general production model.
Qwen3.8-Flash-Next UD-IQ4_XS — ~87.2 GB + MTP 4.14 GB — RESEARCH / REJECT PRODUCTION
Runtime: EngramHalo ROCm, q8 KV, parallel=1, MTP draft Q8_0, n-max=4.
Speed: no MTP 19.49/17.83 tok/s at 8K/24K; MTP 23.06/21.39 (+18–20%). Long run ~10.29 tok/s at 245K, VRAM ~84 GB, soak 99/100.
Quality: 24K marker fails; coding usable content 0/12 or 1/12 depending budget, marker 3/45 maximum. Patched TOP_K/rocPRIM artifact absent. Existing MTP slot test 16/16 at parallel 1 and 16/16 at parallel 4 is build-specific.
IFM K2-Horizon-MoVA-36B-A4B Q4 — ~22.4 GB — REJECT GENERAL PRODUCTION
Separate MBZUAI-IFM fork; quality 6/10, tool/agent soak and serial stability PASS, but Russian/business quality insufficient. Q5/Q8 not tested after Q4 gate.
Nemotron-3-Nano-Omni-30B-A3B Q4 — 22.42 GB + mmproj 1.59 GB — MULTIMODAL EXPERIMENT
Runtime: b10276 ROCm, Reasoning-mmproj-F16, parallel=1. Text/JSON 220/220, stress 1000/1000, tools 15/15, recovery 5/5.
Text: context matrix 7,969/31,950/63,850 — exact 8K FAIL, 32K FAIL, 64K PASS; decode 63.3/59.5/54.3 tok/s.
Early context: token-counted 7,999/31,997/63,993 — 0/3, all returned 0123456789.
Vision: 10/10, wall 1.827 s, prefill 436.5, decode 60.18 tok/s, RAM 5.93 GiB, RSS 2.66 GiB, VRAM 23.6 GiB, GPU 97%. Grounding 1/4. Audio CLI PASS; video abort on patch_embeddings_1.
Qwen3.8-27B UD-Q5_K_XL — 20.88 GB — EXPERIMENTAL CODING SPECIALIST
b10276 ROCm, q8 KV, parallel=1. Coding 12/50; strict JSON 20/20; decode ~10.4 tok/s; soak 42/100; RU/business 36/45; tools 15/15.
32K recall 0/3. Token-counted 64K: 63,993 tokens, 303.9 s, prefill 211.5, decode 9.12 tok/s, strict recall FAIL. Full 50 coding/20 repo gate not complete.
Qwen-AgentWorld-35B-A3B Q4 — 22.13 GB — AGENT SPECIALIST
Vulkan b19cbe9, ctx 65K, q8 KV, parallel=1. Protocol, sandbox, recovery, MCP/web, repo-edit, security/rollback PASS. 50-turn: 50/50 за 93.177 s; security 7/7.
Использовать только в изолированном безопасном sandbox.
Ember + DeepSeek V4 Flash Vision Exp Q2 — 86.72 GB + encoder 0.93 GB — VISION RESEARCH
HSA_USE_SVM=0, HSA_ENABLE_SDMA=0; text 22.43 tok/s. Vision v2 8/10, markers 100%, JSON 80%; coding 1/10, agent 8/10, GUI 0/10, soak 22/100. OCR 4/4 markers and 3/4 JSON; format retry 8/10. Deferred, not production.
GLM-4.7-Flash Q4 — 18.13 GB — EXPERIMENTAL CANDIDATE
b10276 ROCm, ctx 16K, decode ~56.8 tok/s. RU smoke, arithmetic 5/5, tool 1/1, soak 20/20. JSON/code fenced in Markdown; strict output gate remains open. Q6/IQ4 file absent.
Granite 4.2-8B 5.35 GB / LFM2.5-1.2B 0.731 GB — FAST HELPERS
Granite: 38.2–38.9 tok/s, cold full response 4.886 s. LFM: ready 2.216 s, first response 2.407 s, 200 JSON about 211–221 tok/s. Both need verifier and are not production replacements.
Muse Glimmer 30B 16.76 GB + mmproj 1.40 GB — VISION STAGING
b10658 ROCm, text 12.6–12.9 tok/s; vision 5/5, 36.2–118.9 s/image; 1C smoke 194.4 s. Too slow for primary production.
Исторический и неполный inventory
Ранее тестированные модели
| Модель | Результат | Статус |
|---|---|---|
| Qwen3.6 Q2 MTP | 39/42 quality, 100 sequential | Experimental |
| Qwen3-Coder-Next Q4 | ~46.96 tok/s | Coding candidate |
| Gemma 4 12B + mmproj | 25.5–28.18 tok/s, vision 3/3 | Vision specialist |
| Gemma 4 31B + mmproj | 10.6–11.51 tok/s, image PASS | Slow vision |
| MiniCPM5-1B | ~216.7 tok/s targeted smoke; earlier 0/200 | Smoke-only |
| DeepSeek-R1-Distill-32B | 9.1–10.1 short; ~2.5 long | Reject interactive |
| GPT-OSS-120B MXFP4 | ~50–51 tok/s compatible benchmark; pinned runtime failed | Runtime-dependent |
| MiniMax-H3 | 608×352 video 4/4; larger blocked | Video specialist |
Установлены/упоминались, но acceptance нет
Qwen3-8B/14B/32B, Qwen2.5-Coder-32B, Qwen2.5-72B, Llama 3.3 70B, DeepSeek-R1-Distill-Llama-70B, LFM2.5-8B-A1B и Qwen3.8-2.4T-A95B. Metadata или наличие GGUF не означает локальный тест.
Полный дисковый snapshot моделей и проекторов
Snapshot каталога Server 120 от 03.09.2026; наличие файла не означает загрузку в VRAM или acceptance.
| Файл | Размер | Назначение / статус |
|---|---|---|
| qwen2.5-0.5b-instruct-q4_k_m.gguf | 491 MB | Emergency smoke/fallback |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M.gguf | 1.12 GB | Small experimental |
| MiniCPM5-1B-Q4_K_M.gguf | 688 MB | Smoke-plus, acceptance incomplete |
| LFM2.5-1.2B-Instruct-Q4_K_M.gguf | 731 MB | Fast preprocessor |
| mistral-7b-instruct-v0.2.Q4_K_M.gguf | 4.37 GB | Saved candidate |
| Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf | 4.68 GB | Coding candidate |
| gemma-4-12b-it-qat-q4_0.gguf + mmproj | 6.98 + 0.175 GB | Light vision profile |
| T-lite-it-2.1-Q6_K.gguf | 6.72 GB | Candidate |
| Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf | 18.56 GB | Qwen candidate |
| Qwen3.6-27B-Q4_K_M.gguf | 19.10 GB | Dense balanced candidate |
| gemma-4-31B_q4_0-it.gguf + mmproj | 17.65 + 1.20 GB | High-quality vision/manual |
| Qwen3.5-35B-A3B-Revised-q4_k_m.gguf | 21.17 GB | Experimental Qwen |
| DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf | 19.85 GB | Saved, service off |
| T-Search-Q5_K_M.gguf | 25.35 GB | Search/RAG candidate |
| T-pro-it-2.1-Q6_K.gguf | 26.88 GB | Candidate |
| Nemotron Omni MXFP4 + mmproj | 21.73 + 1.59 GB | Historical multimodal experiment |
| Qwen3-Coder-Next-Q4_K_M.gguf | 48.53 GB | Separate coding runtime |
| gpt-oss-120b-MXFP4.gguf | 63.39 GB | Experimental reasoning |
| Whisper large-v3 turbo/q5/q4 | 0.574–1.08 GB | Speech models; runtime acceptance blocked |
| Qwen3.6-35B-A3B-UD-Q2_K_XL.gguf | 12.57 GB | Low-bit/MTP experimental |
| Qwen3.6-35B-A3B-Q4_K_M.gguf | 20.42 GB | Last production baseline; service stopped for acceptance |
Полный reconciliation: SERVER120_MODEL_CATALOG.md.
Открытые пункты
- Patched TOP_K/rocPRIM Flash-Next runtime и ngram-mod A/B.
- Расширенный OpenCode/repo acceptance для Nemotron и полный 50/20 gate Qwen5.
- Nemotron video/GUI action quality — текущие результаты отрицательные.
- GLM Q6/IQ4 и GLM-4.6V — артефакты отсутствуют.
- E-Commerce Bench и 24h thermal soak — отдельные отложенные работы.