GPU池

这份快速指南将引导你如何使用GPU池。

CUDA

https://developer.nvidia.com/cuda-downloads

Runtime

1nvcc -V

Driver

1nvidia-smi

PyTorch

https://pytorch.org/get-started/locally/

install

1pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --trusted-host mirrors.aliyun.com

version

1pip show torch

torch-script

1import torch
2print(torch.__version__)

cuda-script

1import torch
2print(torch.cuda.is_available())
3print(torch.version.cuda)

update

1pip install --upgrade pytorch torchvision --trusted-host mirrors.aliyun.com

uninstall

1pip uninstall torch

硬件

硬件需求

	Bits	7B	13B	30B	65B	8X7B
Full	16 (2 bytes)	160GB	320GB	600GB	1200GB	900GB
Freeze	16 (2 bytes)	20GB	40GB	120GB	240GB	200GB
LoRA	16 (2 bytes)	16GB	32GB	80GB	160GB	120GB
QLoRA	8 (1 bytes)	10GB	16GB	40GB	80GB	80GB
QLoRA	4 (0.5 bytes)	6GB	12GB	24GB	48GB	32GB

计算公式（估算）：显存 = 参数量 x 字节数 (QLoRA 4) = 30,000,000,000 * 0.5 = 15000000000 bytes 15000000000 bytes / 1024 /1024 /1024 = 13.97G 简化为： 15000000000 bytes / 1000 /1000 /1000 = 15G

CUDA

PyTorch

硬件