Can I Run Llama 8B on an RTX 4060? (Yes, Here Is How Well)
Yes, Llama 3.1 8B runs fully on an RTX 4060 8GB at Q4_K_M with about 7 GB used at 8K context. Here are the VRAM numbers, expected speed and the limits.
Yes, Llama 3.1 8B runs fully on an RTX 4060 8GB at Q4_K_M with about 7 GB used at 8K context. Here are the VRAM numbers, expected speed and the limits.