Skip to content

Issues: triton-inference-server/server

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

692 Open 3,253 Closed
692 Open 3,253 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

How to get token usage from openai frontend?
#8194 opened May 8, 2025 by snehalpatel-8451
25.01 vllm tritonserver panic TRITONBACKEND_ResponseFactoryIsCancelled
#8192 opened May 8, 2025 by ColdsteelRail
vLLM backend: Model-specific GPU assignment ignored — both models loaded on GPU 0 despite config.pbtxt specifying gpus: [0] and gpus: [1]
#8189 opened May 7, 2025 by shrivats1995
Docker Image Security Report
#8187 opened May 6, 2025 by Swipe4057
Support deterministic algorithm configuration in PyTorch backend
#8186 opened May 5, 2025 by yhna940
The model instance placement on GPU seems incorrect?
#8184 opened May 5, 2025 by Ind1x1
GPU instances not supported on Jetson Orin AGX 64GB with JetPack 6.2
#8183 opened May 3, 2025 by tiwaojo
Add LoRA metrics compatible with gateway-api-inference-extension
#8181 opened May 1, 2025 by liu-cong
Bug in inception_onnx example model
#8180 opened May 1, 2025 by thanhkiem29
How to set cuda-memory-pool-byte-size and handle the case when we are out of this memory
#8177 opened Apr 29, 2025 by NacerKaciXXII
CLIP Model will not load on CPU-Only Pytorch Build
#8176 opened Apr 28, 2025 by mmtatum
Why throughput is too high when there is only one instance？
#8173 opened Apr 28, 2025 by guanyu-chen-jpg
TensorRT backend multiple optimization profiles support
#8171 opened Apr 27, 2025 by saarus72
Expose IsLastResponse of InferResponse to the Python API
#8169 opened Apr 24, 2025 by danilaeremin
Reschedule request with START and END flag in iterative sequence batching mode
#8167 opened Apr 24, 2025 by danilaeremin
question about scheduling and load distribution
#8166 opened Apr 24, 2025 by Arnold1
use nvcr.io/nvidia/tritonserver:25.03-vllm-python-py3 images deployment qwen2.5-vl-32B-Instruct-AWQ error
#8161 opened Apr 21, 2025 by leimingshuan
How can I use triton core/src /filesystem
#8160 opened Apr 20, 2025 by zjhong12581
1
If I want to implement streaming output for calling OpenAI API, which document should I refer to?
#8157 opened Apr 18, 2025 by zdxff
Feature Request: Support for Dynamic Batching with Variable-Length Inputs in Audio Processing
#8156 opened Apr 18, 2025 by YuBeomGon
Include error code as part of nv_inference_request_failure metric
#8143 opened Apr 11, 2025 by ShuaiShao93
Triton Server uses incorrect batch size
#8141 opened Apr 11, 2025 by saarus72
Deafult value for missing features
#8140 opened Apr 10, 2025 by sumitbinnani
OOM VRAM when using vllm_backend
#8139 opened Apr 10, 2025 by ArtemBiliksin
1
No valid engine configs for ConvFwd_
#8137 opened Apr 9, 2025 by mhbassel
1
ProTip! Follow long discussions with comments:>50.