本文へスキップ
Kitbay
スキル

distributed-training-deepspeed

MIT2026年11月11日 更新33,290GitHubスター

A skill in the claude-code-templates collection, credited to Orchestra Research, with guidance for training large models across GPUs with Microsoft's DeepSpeed library.

スキルの内容

A skill in the claude-code-templates collection, credited to Orchestra Research, with guidance for training large models across GPUs with Microsoft's DeepSpeed library. It says it was generated from DeepSpeed's documentation and covers the ZeRO stages that split optimiser states, gradients and parameters across processes, offloading to CPU memory or NVMe storage, the JSON configuration for batch sizes, optimisers, schedulers and mixed precision, mixture-of-experts models, learning rate range tests, communication-efficient optimisers, pipeline parallelism and the flops profiler. It lists deepspeed, torch, transformers and accelerate as dependencies and notes the DeepSpeed versions some features need. The text is long and mostly condensed documentation.

できること

  • Configure DeepSpeed ZeRO stages
  • Offload model state to CPU or NVMe
  • Write a DeepSpeed JSON configuration
  • Set up mixture-of-experts training