Calculating GPU memory for serving LLMs
In this reference
A practical estimate of GPU memory needs from model size and numeric precision.
Cited in my work
Cite the original work
Calculating GPU memory for serving LLMs (16 November 2023). Substratus. https://www.substratus.ai/blog/calculating-gpu-memory-for-llm
View BibTeX
@misc{calculating_gpu_memory_for_serving_llms,
title = {{Calculating GPU memory for serving LLMs}},
howpublished = {Substratus},
url = {https://www.substratus.ai/blog/calculating-gpu-memory-for-llm},
year = {2023}
}
