本文へスキップ
Kitbay
スキル

simpo

MIT2026年11月11日 更新262,606GitHubスター

An optional Hermes Agent skill, credited to Orchestra Research, for aligning a language model to preference data with SimPO, a method that needs no reference model.

スキルの内容

An optional Hermes Agent skill, credited to Orchestra Research, for aligning a language model to preference data with SimPO, a method that needs no reference model. It walks the agent through a Python 3.10 environment, PyTorch, Hugging Face's alignment-handbook and Flash Attention 2, a YAML training config and a launch with accelerate and DeepSpeed ZeRO-3. It gives settings for a Mistral 7B base model, a Llama 3 8B instruct model and reasoning tasks, explains the main hyperparameters and their ranges, and covers diverging loss, forgotten skills and memory errors. It expects chosen and rejected answer pairs and recommends A100 or H100 GPUs.

できること

  • Set up a SimPO training environment
  • Write a SimPO training config
  • Tune SimPO hyperparameters
  • Troubleshoot preference training runs