本文へスキップ
Kitbay
スキル

accelerate

MIT2026年11月11日 更新262,606GitHubスター

An optional Hermes Agent skill, credited to Orchestra Research, on Hugging Face's Accelerate library for running PyTorch training across several GPUs or machines with few code changes.

スキルの内容

An optional Hermes Agent skill, credited to Orchestra Research, on Hugging Face's Accelerate library for running PyTorch training across several GPUs or machines with few code changes. It shows how to wrap a training script with an Accelerator object, let it handle device placement, and launch on one GPU, many GPUs or several nodes with accelerate config and accelerate launch. It covers fp16, bf16 and fp8 mixed precision, DeepSpeed ZeRO and FSDP sharding with optional CPU offload, gradient accumulation, checkpointing from the main process and seeding for repeatable runs. Common mistakes are listed, and reference files cover Megatron, custom plugins and performance tuning. It runs on Linux, macOS and Windows.

できること

  • Adapt a PyTorch script to Accelerate
  • Launch training on several GPUs or nodes
  • Configure DeepSpeed ZeRO or FSDP sharding
  • Set mixed precision and gradient accumulation