Pular para o conteúdo
Kitbay
Skill

simpo

MITAtualizado em 11 de nov. de 2026262.606Estrelas no GitHub

An optional Hermes Agent skill, credited to Orchestra Research, for aligning a language model to preference data with SimPO, a method that needs no reference model.

O que a skill ensina

An optional Hermes Agent skill, credited to Orchestra Research, for aligning a language model to preference data with SimPO, a method that needs no reference model. It walks the agent through a Python 3.10 environment, PyTorch, Hugging Face's alignment-handbook and Flash Attention 2, a YAML training config and a launch with accelerate and DeepSpeed ZeRO-3. It gives settings for a Mistral 7B base model, a Llama 3 8B instruct model and reasoning tasks, explains the main hyperparameters and their ranges, and covers diverging loss, forgotten skills and memory errors. It expects chosen and rejected answer pairs and recommends A100 or H100 GPUs.

O que faz

  • Set up a SimPO training environment
  • Write a SimPO training config
  • Tune SimPO hyperparameters
  • Troubleshoot preference training runs