Pular para o conteúdo
Kitbay
SkillOficial

hf-cloud-serving-image-selection

Apache-2.0Atualizado em 11 de nov. de 202611.334Estrelas no GitHubInclui scripts

A Hugging Face skill for choosing the serving container when deploying a model to a SageMaker endpoint.

O que a skill ensina

A Hugging Face skill for choosing the serving container when deploying a model to a SageMaker endpoint. It requires a Hugging Face container family (vLLM, vLLM-Omni, SGLang, TEI or PyTorch inference) whenever one can serve the model, gives the few reasons for falling back to a generic image, and reads image URIs from AWS's Deep Learning Containers catalog. It covers choosing between TEI and vLLM for rerankers, AMI and CUDA requirements, container settings, private-network download problems, images known to be broken and first-boot download time. A bundled script mirrors images when a region lacks one.

O que faz

  • Pick the serving container for a model on SageMaker
  • Look up current image URIs from AWS's container catalog
  • Configure vLLM and TEI containers
  • Mirror an image to another region