tensorrt-llm
An optional Hermes Agent skill, credited to Orchestra Research, on serving large language models with NVIDIA's TensorRT-LLM library on NVIDIA GPUs under Linux.
What the skill teaches
An optional Hermes Agent skill, credited to Orchestra Research, on serving large language models with NVIDIA's TensorRT-LLM library on NVIDIA GPUs under Linux. It says when to choose it over vLLM or llama.cpp, how to install it from NGC Docker images or pip with the required CUDA, TensorRT and Python versions, and how to use the Python LLM API, the trtllm-serve server and its OpenAI-style chat endpoint. Patterns cover FP8 and other quantisation, tensor parallelism across GPUs and in-flight batching, with listed model families and features such as speculative decoding and LoRA serving. Three reference files hold optimisation, multi-GPU and production serving notes. Its throughput figures are the library's own claims.
Capabilities
- Install TensorRT-LLM from NGC or pip
- Serve a model with trtllm-serve
- Load quantised models in FP8
- Spread a model across several GPUs
About
- Publisher
- Nous Research
Install
The project doesn't document install steps that we could confirm. See the project's own instructions
What's inside
- references/multi-gpu.md
- references/optimization.md
- references/serving.md
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Not stated in its documentation | |
| Publisher | Nous Research | source |
| Official status | Community (not published by the tool's vendor) | source |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | No | source |
| Platforms | Linux | source |
| AI clients | Not stated in its documentation | |
| Licence | MIT | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 11, 2026262,606 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository Website |
Sources
- Skill file (SKILL.md)github.com/nousresearch/hermes-agent · Backs: Licence, Official status, PlatformsNov 17, 2026
- GitHub repositorygithub.com/nousresearch/hermes-agent · Backs: Publisher, Activity, LicenceNov 11, 2026
- Folder in the repository (tensorrt-llm)github.com/nousresearch/hermes-agent · Backs: Includes scriptsNov 17, 2026
- Skill file (SKILL.md)github.com/NousResearch/hermes-agentNov 10, 2026
- Folder in the repository (tensorrt-llm)github.com/NousResearch/hermes-agent · Backs: Includes scriptsNov 10, 2026