Quantize Llama models with GGUF and llama.cpp
In this reference
A guide to quantizing models for use with llama.cpp.
Cite the original work
Maxime Labonne. Quantize Llama models with GGUF and llama.cpp. https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html
View BibTeX
@misc{quantize_llama_models_with_gguf_and_llama_cpp,
author = {Maxime Labonne},
title = {{Quantize Llama models with GGUF and llama.cpp}},
howpublished = {Maxime Labonne},
url = {https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html}
}
