GGUF: What's Inside Besides Weights, and What's Still Missing?

If you've ever downloaded an LLM and thought "great, another format," you know GGUF. But what's actually inside that file besides the weights? Turns out, it's a zoo of metadata — from data types to tokenizer info. But like any zoo, some cages are empty.

GGUF is the evolution of GGML, designed to load models without the usual dance. Inside: a header, key-value metadata, and tensors. Metadata can include everything from model name to tokenization details. But here's the rub: the standard still hasn't defined a bunch of important stuff — like how to store LoRA adapter info or how to specify custom architectures. Developers hack around it: some shove JSON into metadata, others use separate files. The irony: a format meant to simplify life sometimes turns into "guess the architecture" minigame.

What's truly missing? First, a clear spec for describing architectures — right now it's a creative field bordering on chaos. Second, support for training checkpoints (yes, GGUF is inference-focused, but a dev can dream). Third, a unified way to store preprocessors — without it, every framework does its own magic, and the user suffers.

METABYTE studio comment: GGUF is a great idea, but like any young standard, it needs some elbow grease. In our projects, we prefer wrapping models in universal containers so we don't have to guess which tokenizer was used. Are you still keeping weights in Safetensors?