Two numbers determine whether a vLLM deployment is healthy: KV cache usage and prefix cache hit rate. This post explains what they measure, how vLLM computes them from its block pool, and what the LRU evictor does when memory runs out.
Two numbers determine whether a vLLM deployment is healthy: KV cache usage and prefix cache hit rate. This post explains what they measure, how vLLM computes them from its block pool, and what the LRU evictor does when memory runs out.
Perplexity, negative log-likelihood, entropy, KL divergence, and cross-entropy annotated term-by-term — what each measures, how they relate, and what values to expect during training.
Perplexity, negative log-likelihood, entropy, KL divergence, and cross-entropy annotated term-by-term — what each measures, how they relate, and what values to expect during training.
How do you measure whether a model is actually good? The answer is a set of metrics — precision, recall, F1, perplexity, calibration, confidence intervals — each measuring something different and failing in a different way.
How do you measure whether a model is actually good? The answer is a set of metrics — precision, recall, F1, perplexity, calibration, confidence intervals — each measuring something different and failing in a different way.