torchtitan
An optional skill in Nous Research's Hermes Agent, credited to Orchestra Research, for pretraining large language models with TorchTitan, PyTorch's platform that combines data, tensor, pipeline and context parallelism.
What the skill teaches
An optional skill in Nous Research's Hermes Agent, credited to Orchestra Research, for pretraining large language models with TorchTitan, PyTorch's platform that combines data, tensor, pipeline and context parallelism. It covers installing the package, fetching a tokenizer from Hugging Face, choosing named run configurations with command-line overrides, launching an eight-GPU single-node run, writing a SLURM script for multi-node jobs that resume from checkpoints, and enabling Float8 training. For very large models it explains creating a seed checkpoint before splitting work into pipeline stages. Troubleshooting notes cover memory errors and checkpoint resharding, and reference files describe FSDP2, Float8 recipes, checkpoints and adding custom models.
Capabilities
- Launch single-node TorchTitan pretraining
- Write multi-node SLURM training jobs
- Enable Float8 training
- Combine data, tensor and pipeline parallelism
About
- Publisher
- Nous Research
Install
The project doesn't document install steps that we could confirm. See the project's own instructions
What's inside
- references/checkpoint.md
- references/custom-models.md
- references/float8.md
- references/fsdp.md
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Not stated in its documentation | |
| Publisher | Nous Research | source |
| Official status | Not stated in its documentation | |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | No | source |
| Platforms | Linux, macOS | source |
| AI clients | Not stated in its documentation | |
| Licence | MIT | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 11, 2026262,606 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository Website |
Sources
- Skill file (SKILL.md)github.com/nousresearch/hermes-agent · Backs: Licence, PlatformsNov 16, 2026
- GitHub repositorygithub.com/nousresearch/hermes-agent · Backs: Publisher, Activity, LicenceNov 11, 2026
- GitHub API recordgithub.com/nousresearch/hermes-agent · Backs: Includes scriptsNov 16, 2026
- Skill file (SKILL.md)github.com/NousResearch/hermes-agentNov 10, 2026
- Folder in the repository (torchtitan)github.com/NousResearch/hermes-agent · Backs: Includes scriptsNov 10, 2026