{
  "module": "FTDD-09 — llama.cpp + vLLM",
  "course": "3 — LLM Fine-Tuning Masterclass",
  "version": "1.0.0",
  "duration_minutes": 45,
  "total_questions": 10,
  "bloom_distribution": {
    "target": "40% recall / 30% application / 30% analysis",
    "actual": { "recall": 4, "application": 3, "analysis": 3 }
  },
  "passing_score_percent": 70,
  "questions": [
    {
      "id": "Q01", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is llama.cpp's defining property?",
      "options": [
        "Maximum throughput on GPUs under concurrent load.",
        "Hardware breadth: a single C/C++ binary running on CPU, Metal, CUDA, ROCm, and Vulkan.",
        "Native support for AWQ and GPTQ quantization.",
        "A Python-based serving framework with an OpenAI-compatible API."
      ],
      "answer_index": 1,
      "rationale": "llama.cpp's defining property is hardware breadth. The same C/C++ binary (no Python runtime) runs on CPU, Apple Metal, NVIDIA CUDA, AMD ROCm, and Vulkan. If hardware exists, llama.cpp can probably run on it. This makes it the tool for laptops, edge boxes, air-gapped servers, and Macs."
    },
    {
      "id": "Q02", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What are the two innovations that make vLLM win production throughput?",
      "options": [
        "GGUF and k-quants.",
        "Tensor parallelism and FP8 quantization.",
        "PagedAttention (KV cache as virtual memory) and continuous batching (admit requests mid-generation).",
        "The OpenAI-compatible API and a single static binary."
      ],
      "answer_index": 2,
      "rationale": "PagedAttention manages the KV cache like virtual memory (fixed-size blocks, allocated on demand), eliminating fragmentation so many more concurrent requests fit per GPU. Continuous batching admits new requests at the iteration/token level as slots free. Together they keep the GPU saturated and memory utilized. (Kwon et al., SOSP 2023.)"
    },
    {
      "id": "Q03", "bloom": "recall", "type": "multiple_choice",
      "prompt": "Which quant format does llama.cpp use, and what are its quant schemes called?",
      "options": [
        "AWQ; activation-aware schemes.",
        "GGUF; k-quants (Q4_K_M, Q5_K_M, Q8_0).",
        "FP8; native 8-bit float.",
        "GPTQ; post-training 4-bit."
      ],
      "answer_index": 1,
      "rationale": "GGUF is llama.cpp's native container (one file: weights, tokenizer, metadata). Its quantization produces k-quants — Q4_K_M, Q5_K_M, Q8_0 — block-wise schemes that adapt precision per weight block. Q4_K_M is the widely-recommended quality/size default."
    },
    {
      "id": "Q04", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is llama.cpp's telemetry posture, and why does it matter?",
      "options": [
        "It sends anonymous usage stats by default, which can be disabled.",
        "It makes no network calls and collects no telemetry — a single static binary droppable on an internet-isolated machine. This makes it the air-gap default for HIPAA/government/classified (Pillar 7).",
        "It requires a license server connection on startup.",
        "It phones home for model updates."
      ],
      "answer_index": 1,
      "rationale": "llama.cpp makes no network calls and collects no telemetry. A single static binary you can drop on a machine that has never touched the internet. For HIPAA, government, and classified environments, a server that phones home is a non-starter. The single binary = minimal assurance surface."
    },
    {
      "id": "Q05", "bloom": "application", "type": "multiple_choice",
      "prompt": "A team quantized their fine-tuned model to AWQ, then discovered they must deploy to an air-gapped CPU-only box. What went wrong, and what must they do?",
      "options": [
        "Nothing — AWQ runs on CPU in llama.cpp.",
        "They chose the quant before the server. AWQ is a vLLM/GPU format; the CPU box needs llama.cpp, which requires GGUF. They must re-export the trained weights to GGUF (e.g., Q4_K_M).",
        "They need to install vLLM on the CPU box.",
        "They need to convert the AWQ to FP8."
      ],
      "answer_index": 1,
      "rationale": "You cannot cross-serve: GGUF runs in llama.cpp, AWQ/GPTQ/FP8 run in vLLM. The quant choice is downstream of the serving choice. Decide the server first (by deployment context — CPU/air-gap = llama.cpp), then export to its format (GGUF). The team must re-export to GGUF."
    },
    {
      "id": "Q06", "bloom": "application", "type": "multiple_choice",
      "prompt": "You are deploying a cloud API serving hundreds of concurrent users on NVIDIA GPUs. Which engine and quant format?",
      "options": [
        "llama.cpp with GGUF Q4_K_M.",
        "vLLM with AWQ (or FP16 if VRAM allows). vLLM's PagedAttention + continuous batching maximize throughput under concurrent load.",
        "TensorRT-LLM with GGUF.",
        "llama.cpp with FP8."
      ],
      "answer_index": 1,
      "rationale": "GPU datacenter with concurrent users and throughput as the constraint = vLLM's regime. AWQ or GPTQ (or FP16/BF16 if VRAM allows) are vLLM's formats. PagedAttention and continuous batching are why vLLM wins production throughput. Start here; consider TensorRT-LLM only with measured reason."
    },
    {
      "id": "Q07", "bloom": "application", "type": "multiple_choice",
      "prompt": "Your team is building a local desktop assistant that must run on users' laptops, which may or may not have a GPU. Which engine, and why?",
      "options": [
        "vLLM — it has an OpenAI-compatible API.",
        "llama.cpp — its hardware breadth means the same binary runs on CPU, Metal, or CUDA depending on what the laptop has. Single self-contained binary, no Python runtime.",
        "TensorRT-LLM — lowest latency.",
        "SGLang — best for structured output."
      ],
      "answer_index": 1,
      "rationale": "A user laptop may not have a GPU, and the binary must be self-contained. llama.cpp's defining property is hardware breadth: the same binary runs on CPU, Metal, CUDA, ROCm. This is the long-tail/edge deployment target where llama.cpp is the tool. vLLM is GPU-oriented; TensorRT-LLM adds complexity unjustified here."
    },
    {
      "id": "Q08", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why does PagedAttention increase the number of concurrent requests a GPU can serve?",
      "options": [
        "It compresses the model weights to 4-bit.",
        "It eliminates KV-cache fragmentation. Naive serving pre-allocates contiguous blocks sized for max sequence length, wasting memory on requests that finish early. PagedAttention allocates fixed-size blocks on demand, so memory is only used for tokens actually generated — fitting more concurrent requests.",
        "It parallelizes the model across more GPUs.",
        "It skips the KV cache entirely."
      ],
      "answer_index": 1,
      "rationale": "The KV cache stores keys/values per generated token. Naive pre-allocation fragments memory because most requests don't use their full max-length allocation. PagedAttention (virtual-memory style: fixed-size blocks, on-demand allocation) eliminates the waste, so the same GPU holds more concurrent requests' caches — directly raising throughput."
    },
    {
      "id": "Q09", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why is the statement 'GGUF and AWQ are interchangeable quant formats' wrong, and what is the correct relationship?",
      "options": [
        "They are interchangeable — both run in any engine.",
        "They are tied to different engines' kernels: GGUF runs only in llama.cpp; AWQ runs only in vLLM. You cannot cross-serve. The quant choice is downstream of the serving choice, not independent.",
        "GGUF is newer and replaces AWQ.",
        "AWQ is the CPU version of GGUF."
      ],
      "answer_index": 1,
      "rationale": "The formats are tied to their engines' kernels. GGUF is llama.cpp's format (k-quants, designed for heterogeneous hardware). AWQ/GPTQ/FP8 are vLLM's formats (GPU-native kernels). You cannot serve a GGUF in vLLM or an AWQ in llama.cpp. The server is chosen by deployment context; the quant format follows."
    },
    {
      "id": "Q10", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why do most teams start at vLLM rather than jumping to TensorRT-LLM or SGLang for production GPU serving?",
      "options": [
        "Because vLLM is the only engine with an OpenAI-compatible API.",
        "Because vLLM offers strong throughput via PagedAttention and continuous batching with a manageable Python-based setup and OpenAI-compatible API. TensorRT-LLM adds compilation/tuning complexity; SGLang is specialized. Moving up is justified only with measured data that vLLM is the bottleneck.",
        "Because TensorRT-LLM and SGLang are deprecated.",
        "Because vLLM is always faster than the alternatives."
      ],
      "answer_index": 1,
      "rationale": "vLLM is the production GPU default because it delivers strong throughput (PagedAttention, continuous batching, tensor parallelism) with a manageable setup and OpenAI-compatible API. TensorRT-LLM's extra compilation work and SGLang's specialization are justified only when you have measured that vLLM is your bottleneck. Start at vLLM; move up with data."
    }
  ]
}