hanxiao/Qwen3.6-35B-A3B-MTP-L4

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Docker image. Decode-optimized to ~91–99 tok/s (min ~91 chat, max ~99 math), lossless (full GPU residency + ECC-off).

语言：Shell · GitHub 星标：19 · Fork：2

话题：gcp · gguf · gpu-inference · inference-optimization · large-language-models · llama-cpp · llm · local-llm

在 GitHub 上查看原仓库