Calculating GPU memory for serving LLMs
A practical estimate of GPU memory needs from model size and numeric precision.
Substratus16 November 2023Public source
Read the archived source
In this reference
A practical estimate of GPU memory needs from model size and numeric precision.
Cited in my work
Cite the original work
Calculating GPU memory for serving LLMs (16 November 2023). Substratus. https://www.substratus.ai/blog/calculating-gpu-memory-for-llm
View BibTeX
@misc{calculating_gpu_memory_for_serving_llms,
title = {{Calculating GPU memory for serving LLMs}},
howpublished = {Substratus},
url = {https://www.substratus.ai/blog/calculating-gpu-memory-for-llm},
year = {2023}
}
