Zum Inhalt springen
Kitbay
Skill

accelerate

MITAktualisiert am 11. Nov. 2026262.606GitHub-Sterne

An optional Hermes Agent skill, credited to Orchestra Research, on Hugging Face's Accelerate library for running PyTorch training across several GPUs or machines with few code changes.

Was der Skill vermittelt

An optional Hermes Agent skill, credited to Orchestra Research, on Hugging Face's Accelerate library for running PyTorch training across several GPUs or machines with few code changes. It shows how to wrap a training script with an Accelerator object, let it handle device placement, and launch on one GPU, many GPUs or several nodes with accelerate config and accelerate launch. It covers fp16, bf16 and fp8 mixed precision, DeepSpeed ZeRO and FSDP sharding with optional CPU offload, gradient accumulation, checkpointing from the main process and seeding for repeatable runs. Common mistakes are listed, and reference files cover Megatron, custom plugins and performance tuning. It runs on Linux, macOS and Windows.

Funktionen

  • Adapt a PyTorch script to Accelerate
  • Launch training on several GPUs or nodes
  • Configure DeepSpeed ZeRO or FSDP sharding
  • Set mixed precision and gradient accumulation