Module: FTDD-09 — llama.cpp + vLLM Diagram count: 4 Tool: Mermaid (primary). Each diagram validated in Mermaid Live Editor.
Type: Decision / comparison Purpose: llama.cpp and vLLM are complementary, not competing. The deployment context decides. Reading the diagram: Follow the question at the top. The left branch is llama.cpp's regime; the right is vLLM's. Many teams use both.
flowchart TD
Q{"DEPLOYMENT CONTEXT?\nWhat hardware, what network,\nwhat traffic profile?"}
Q -->|"CPU / Mac / heterogeneous /\nair-gapped / single binary"| LCPP["llama.cpp\nsingle C/C++ binary\nno Python runtime\nmax hardware breadth\nair-gap default"]
Q -->|"GPU datacenter /\nconcurrent users /\nthroughput-bound"| VLLM["vLLM\nPython GPU engine\nPagedAttention\ncontinuous batching\nproduction default"]
Both["MANY TEAMS USE BOTH:\nllama.cpp for local/edge\nvLLM for cloud service"]
LCPP --> Both
VLLM --> Both
style Q fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4
style LCPP fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style VLLM fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style Both fill:#08080c,stroke:rgba(255,255,255,0.12),color:#9494a0
Type: Mapping / hard constraint Purpose: GGUF runs in llama.cpp. AWQ/GPTQ/FP8 run in vLLM. You CANNOT cross-serve. The quant choice is downstream of the serving choice. Reading the diagram: Each server shows the formats it accepts. The crossed-out arrows show the forbidden combinations.
flowchart LR
subgraph LCPP["llama.cpp"]
G["GGUF\n(k-quants: Q4_K_M,\nQ5_K_M, Q8_0)"]
end
subgraph VLLM["vLLM"]
A["AWQ"]
G2["GPTQ"]
F["FP8 (Hopper/Blackwell)"]
FP["FP16 / BF16 (no quant)"]
end
GGUFx["GGUF in vLLM\nFORBIDDEN"]
AWQx["AWQ in llama.cpp\nFORBIDDEN"]
G -.x.-> VLLM
A -.x.-> LCPP
G --> GGUFx
A --> AWQx
style G fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style A fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style G2 fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style F fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style FP fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style GGUFx fill:#08080c,stroke:rgba(255,255,255,0.08),color:#9494a0
style AWQx fill:#08080c,stroke:rgba(255,255,255,0.08),color:#9494a0
Type: Mechanism / process Purpose: PagedAttention (KV cache as virtual memory) + continuous batching (admit mid-generation) are why vLLM is the production GPU default. Reading the diagram: Top = the two mechanisms. Bottom = the combined effect on throughput.
flowchart TD
subgraph PA["PAGEDATTENTION (Kwon et al., SOSP 2023)"]
PA1["KV cache managed like VIRTUAL MEMORY\nfixed-size blocks (pages)\nallocated on demand\nNO fragmentation waste"]
end
subgraph CB["CONTINUOUS BATCHING"]
CB1["admit new requests at the\nITERATION / TOKEN level\nas slots free\nGPU stays saturated"]
end
PA --> Effect["COMBINED EFFECT:\nmany more concurrent requests per GPU\nhigh aggregate throughput\nGPU memory fully utilized"]
CB --> Effect
Effect --> API["OpenAI-compatible HTTP API\nany OpenAI SDK client works unchanged"]
style PA1 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style CB1 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style Effect fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style API fill:#08080c,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
Type: Tiered spectrum Purpose: Where each engine sits. llama.cpp for breadth/air-gap; vLLM for the production default; SGLang and TensorRT-LLM for the high end. Reading the diagram: Bottom to top = increasing optimization for specific high-end workloads, at the cost of more setup complexity. Start at vLLM; move up with measured reason.
flowchart TD
TRT["TENSORRT-LLM (NVIDIA)\nlowest latency, highest throughput\nNVIDIA-only, extra compilation work\nTHE HIGH-END NVIDIA TIER"]
SGL["SGLang\nRadixAttention prefix caching\nstructured generation\nEXCEL on shared-prefix / constrained output"]
VLLM["vLLM\nproduction GPU default\nPagedAttention + continuous batching\nOpenAI-compatible API"]
LCPP["llama.cpp\nmax hardware breadth\nair-gap default\nTHE UNIVERSAL SINGLE-BINARY"]
LCPP -->|"more throughput-optimized,\nmore setup complexity"| VLLM
VLLM -->|"move up with MEASURED reason"| SGL
VLLM -->|"move up with MEASURED reason\n(NVIDIA hardware)"| TRT
Rule["MOST TEAMS START AT vLLM.\nMove up only when you have data\nshowing vLLM is your bottleneck."]
VLLM --> Rule
style TRT fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style SGL fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style VLLM fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
style LCPP fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style Rule fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4
#14141f panel fill, #5eead4 accent for primary, rgba(94,234,212,0.5) for secondary borders, #e4e4e8 / #9494a0 for text.flowchart) supported in current Mermaid (v10.4+).# Diagrams — Module FTDD-09: llama.cpp + vLLM
**Module**: FTDD-09 — llama.cpp + vLLM
**Diagram count**: 4
**Tool**: Mermaid (primary). Each diagram validated in [Mermaid Live Editor](https://mermaid.live).
---
## Diagram 1 — Two Tools, Two Jobs (the decision)
**Type**: Decision / comparison
**Purpose**: llama.cpp and vLLM are complementary, not competing. The deployment context decides.
**Reading the diagram**: Follow the question at the top. The left branch is llama.cpp's regime; the right is vLLM's. Many teams use both.
```mermaid
flowchart TD
Q{"DEPLOYMENT CONTEXT?\nWhat hardware, what network,\nwhat traffic profile?"}
Q -->|"CPU / Mac / heterogeneous /\nair-gapped / single binary"| LCPP["llama.cpp\nsingle C/C++ binary\nno Python runtime\nmax hardware breadth\nair-gap default"]
Q -->|"GPU datacenter /\nconcurrent users /\nthroughput-bound"| VLLM["vLLM\nPython GPU engine\nPagedAttention\ncontinuous batching\nproduction default"]
Both["MANY TEAMS USE BOTH:\nllama.cpp for local/edge\nvLLM for cloud service"]
LCPP --> Both
VLLM --> Both
style Q fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4
style LCPP fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style VLLM fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style Both fill:#08080c,stroke:rgba(255,255,255,0.12),color:#9494a0
```
---
## Diagram 2 — Quant-Format Compatibility (formats follow the server)
**Type**: Mapping / hard constraint
**Purpose**: GGUF runs in llama.cpp. AWQ/GPTQ/FP8 run in vLLM. You CANNOT cross-serve. The quant choice is downstream of the serving choice.
**Reading the diagram**: Each server shows the formats it accepts. The crossed-out arrows show the forbidden combinations.
```mermaid
flowchart LR
subgraph LCPP["llama.cpp"]
G["GGUF\n(k-quants: Q4_K_M,\nQ5_K_M, Q8_0)"]
end
subgraph VLLM["vLLM"]
A["AWQ"]
G2["GPTQ"]
F["FP8 (Hopper/Blackwell)"]
FP["FP16 / BF16 (no quant)"]
end
GGUFx["GGUF in vLLM\nFORBIDDEN"]
AWQx["AWQ in llama.cpp\nFORBIDDEN"]
G -.x.-> VLLM
A -.x.-> LCPP
G --> GGUFx
A --> AWQx
style G fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style A fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style G2 fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style F fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style FP fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style GGUFx fill:#08080c,stroke:rgba(255,255,255,0.08),color:#9494a0
style AWQx fill:#08080c,stroke:rgba(255,255,255,0.08),color:#9494a0
```
---
## Diagram 3 — Why vLLM Wins Production Throughput
**Type**: Mechanism / process
**Purpose**: PagedAttention (KV cache as virtual memory) + continuous batching (admit mid-generation) are why vLLM is the production GPU default.
**Reading the diagram**: Top = the two mechanisms. Bottom = the combined effect on throughput.
```mermaid
flowchart TD
subgraph PA["PAGEDATTENTION (Kwon et al., SOSP 2023)"]
PA1["KV cache managed like VIRTUAL MEMORY\nfixed-size blocks (pages)\nallocated on demand\nNO fragmentation waste"]
end
subgraph CB["CONTINUOUS BATCHING"]
CB1["admit new requests at the\nITERATION / TOKEN level\nas slots free\nGPU stays saturated"]
end
PA --> Effect["COMBINED EFFECT:\nmany more concurrent requests per GPU\nhigh aggregate throughput\nGPU memory fully utilized"]
CB --> Effect
Effect --> API["OpenAI-compatible HTTP API\nany OpenAI SDK client works unchanged"]
style PA1 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style CB1 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style Effect fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style API fill:#08080c,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
```
---
## Diagram 4 — The Serving Tiers
**Type**: Tiered spectrum
**Purpose**: Where each engine sits. llama.cpp for breadth/air-gap; vLLM for the production default; SGLang and TensorRT-LLM for the high end.
**Reading the diagram**: Bottom to top = increasing optimization for specific high-end workloads, at the cost of more setup complexity. Start at vLLM; move up with measured reason.
```mermaid
flowchart TD
TRT["TENSORRT-LLM (NVIDIA)\nlowest latency, highest throughput\nNVIDIA-only, extra compilation work\nTHE HIGH-END NVIDIA TIER"]
SGL["SGLang\nRadixAttention prefix caching\nstructured generation\nEXCEL on shared-prefix / constrained output"]
VLLM["vLLM\nproduction GPU default\nPagedAttention + continuous batching\nOpenAI-compatible API"]
LCPP["llama.cpp\nmax hardware breadth\nair-gap default\nTHE UNIVERSAL SINGLE-BINARY"]
LCPP -->|"more throughput-optimized,\nmore setup complexity"| VLLM
VLLM -->|"move up with MEASURED reason"| SGL
VLLM -->|"move up with MEASURED reason\n(NVIDIA hardware)"| TRT
Rule["MOST TEAMS START AT vLLM.\nMove up only when you have data\nshowing vLLM is your bottleneck."]
VLLM --> Rule
style TRT fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style SGL fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
style VLLM fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
style LCPP fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
style Rule fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4
```
---
## Validation notes
- All four diagrams use the course design system colors: `#14141f` panel fill, `#5eead4` accent for primary, `rgba(94,234,212,0.5)` for secondary borders, `#e4e4e8` / `#9494a0` for text.
- Paste each into [Mermaid Live Editor](https://mermaid.live) to render. All use stable Mermaid syntax (`flowchart`) supported in current Mermaid (v10.4+).
- For the slide deck (artifact 03), these are rendered as static SVG/PNG captures from Mermaid Live, inlined into reveal.js.