Zum Inhalt springen
Kitbay
Skill

simpo

MITAktualisiert am 11. Nov. 2026262.606GitHub-Sterne

An optional Hermes Agent skill, credited to Orchestra Research, for aligning a language model to preference data with SimPO, a method that needs no reference model.

Was der Skill vermittelt

An optional Hermes Agent skill, credited to Orchestra Research, for aligning a language model to preference data with SimPO, a method that needs no reference model. It walks the agent through a Python 3.10 environment, PyTorch, Hugging Face's alignment-handbook and Flash Attention 2, a YAML training config and a launch with accelerate and DeepSpeed ZeRO-3. It gives settings for a Mistral 7B base model, a Llama 3 8B instruct model and reasoning tasks, explains the main hyperparameters and their ranges, and covers diverging loss, forgotten skills and memory errors. It expects chosen and rejected answer pairs and recommends A100 or H100 GPUs.

Funktionen

  • Set up a SimPO training environment
  • Write a SimPO training config
  • Tune SimPO hyperparameters
  • Troubleshoot preference training runs