vLLM issues | Ecosyste.ms: OpenCollective

Add classifiers in setup.py

github.com/vllm-project/vllm - terrytangyuan opened this pull request 9 days ago

[Doc] Fix VLM prompt placeholder sample bug

github.com/vllm-project/vllm - ycool opened this pull request 10 days ago

[Usage]: due to large max_mm_tokens, number of images that multimodal models can support is underestimated

github.com/vllm-project/vllm - SepehrV opened this issue 10 days ago

[Bug]: vLLM OpenAI-api server `/docs` endpoint fails to load

github.com/vllm-project/vllm - mgoin opened this issue 10 days ago

[Misc] Improve validation errors around best_of and n

github.com/vllm-project/vllm - tjohnson31415 opened this pull request 10 days ago

[WIP] Prototyping re-arch

github.com/vllm-project/vllm - WoosukKwon opened this pull request 10 days ago

[ci][test] use load dummy for testing

github.com/vllm-project/vllm - youkaichao opened this pull request 10 days ago

[Feature]: Enabling MSS for larger number of sequences (>256)

github.com/vllm-project/vllm - kushanam opened this issue 10 days ago

[Performance]: Llama-3.2-11B-Vision-Instruct taking up a lot of memory

github.com/vllm-project/vllm - pbarker opened this issue 10 days ago

mypy: check additional directories

github.com/vllm-project/vllm - russellb opened this pull request 10 days ago

Add `lm-eval` directly to requirements-test.txt

github.com/vllm-project/vllm - mgoin opened this pull request 10 days ago

[Bugfix] Optimize composite weight loading and fix EAGLE weight loading

github.com/vllm-project/vllm - DarkLight1337 opened this pull request 10 days ago

[Bugfix][Doc] Report neuron error in output

github.com/vllm-project/vllm - joerowell opened this pull request 10 days ago

[Misc]: How to set num-scheduler-steps

github.com/vllm-project/vllm - o1iv3r opened this issue 10 days ago

[Usage]: Multi-gpu inference takes too much memory + how to make uneven load

github.com/vllm-project/vllm - Ouna-the-Dataweaver opened this issue 10 days ago

[Misc]: Segmentation Fault in vLLM API Server during Model Initialization (NCCL Error: Unhandled System Error)

github.com/vllm-project/vllm - shreyasp-07 opened this issue 10 days ago

[Doc] Update vlm.rst to include an example on videos

github.com/vllm-project/vllm - sayakpaul opened this pull request 10 days ago

[Frontend][Feature] Add jamba tool parser

github.com/vllm-project/vllm - tomeras91 opened this pull request 10 days ago

[Bug]: InternVL bounding box prediction does not work

github.com/vllm-project/vllm - MoritzLaurer opened this issue 10 days ago

[Bug]: Can not pip install vllm inside docker

github.com/vllm-project/vllm - fahadh4ilyas opened this issue 10 days ago

[Frontend] Add Early Validation For Chat Template / Tool Call Parser

github.com/vllm-project/vllm - alex-jw-brooks opened this pull request 10 days ago

[Misc]: Nobody reviews my PR

github.com/vllm-project/vllm - CharlesRiggins opened this issue 10 days ago

[Core] Add an environment variable which needs to be set explicitly to allow BlockSpaceManagerV1

github.com/vllm-project/vllm - sroy745 opened this pull request 10 days ago

support bitsandbytes quantization with more models

github.com/vllm-project/vllm - chenqianfzh opened this pull request 10 days ago

[Neuron] Introduce paged attention support for neuron backend

github.com/vllm-project/vllm - liangfu opened this pull request 10 days ago

[Bug]: vllm much slower on long context inputs when using --enable-lora even when lora is not used

github.com/vllm-project/vllm - badrjd opened this issue 10 days ago

[Bugfix] Fix crashing for multimodal when image passed with height == 1

github.com/vllm-project/vllm - Pernekhan opened this pull request 10 days ago

[torch.compile] Fuse RMSNorm with quant

github.com/vllm-project/vllm - ProExpertProg opened this pull request 11 days ago

[Doc] Improve contributing and installation documentation

github.com/vllm-project/vllm - rafvasq opened this pull request 11 days ago

[Core][Frontend] Add Support for Inference Time mm_processor_kwargs

github.com/vllm-project/vllm - alex-jw-brooks opened this pull request 11 days ago

[CI/Build] Update Dockerfile install+deploy image to ubuntu 22.04

github.com/vllm-project/vllm - mgoin opened this pull request 11 days ago

[Bug]: assert len(indices) == len(inputs) with `Qwen/Qwen2-VL-2B-Instruct`

github.com/vllm-project/vllm - sayakpaul opened this issue 11 days ago

[Bug]: Error Encountered in vLLM Benchmarking with Input Length greater than 8192 in Llama 3.1 405B Model

github.com/vllm-project/vllm - Bihan opened this issue 11 days ago

[Usage]: Not getting the infrence metrics in the api response

github.com/vllm-project/vllm - vverma01232 opened this issue 11 days ago

[New Model]: silma-ai/SILMA-9B-Instruct-v1.0

github.com/vllm-project/vllm - hassanraha opened this issue 11 days ago

[Core]: (1/N) Support prefill only models by Workflow Defined Engine - Prefill only attention

github.com/vllm-project/vllm - noooop opened this pull request 11 days ago

[Bugfix][Core] Handle empty ids_list in BlockSpaceManagerV1.get_common_computed_block_ids to prevent msgspec serialization errors

github.com/vllm-project/vllm - amberOoO opened this pull request 11 days ago

[Bug] BlockSpaceManagerV1.get_common_computed_block_ids returns empty string, causing msgspec decode failure

github.com/vllm-project/vllm - amberOoO opened this issue 11 days ago

[OpenVINO] Use torch 2.4.0 and newer optimim version

github.com/vllm-project/vllm - ilya-lavrenov opened this pull request 11 days ago

[Bug]: Unsupported base layer: QKVParallelLinear when loading lora to a quantized model

github.com/vllm-project/vllm - fahadh4ilyas opened this issue 11 days ago

[Bug]: Installation from last commit (version wrong)

github.com/vllm-project/vllm - johnnynunez opened this issue 11 days ago

[Bug]: Issue Running VLLM Open AI using nonroot user in K8s

github.com/vllm-project/vllm - luhurfth opened this issue 11 days ago

[Frontend] API support for beam search for MQLLMEngine

github.com/vllm-project/vllm - LunrEclipse opened this pull request 11 days ago

[Bugfix][Hardware] Fix model input for decode

github.com/vllm-project/vllm - yma11 opened this pull request 11 days ago

[Usage]: How to run llama 3.2 with CPU only version

github.com/vllm-project/vllm - chanandrew96 opened this issue 11 days ago

[Bug] In v0.6.2, when tp=1, TPOT becomes very slow for batch sizes of 10 or so. (not happened in v0.5.5)

github.com/vllm-project/vllm - ashgold opened this issue 11 days ago

[Feature]: Support for ONNX

github.com/vllm-project/vllm - LetianLee opened this issue 11 days ago

[Bug]: AMD MultiStep Feature Issue. Missing argument: 'turn_prefills_into_decodes' in `advance_step()`

github.com/vllm-project/vllm - tjtanaa opened this issue 11 days ago

[Feature]: LLMEngine and ModelConfig explicitly require path or HF model id, but no InferenceClient class for locally running VLLM server

github.com/vllm-project/vllm - DanielViglione opened this issue 12 days ago

support jetson AGX Orin

github.com/vllm-project/vllm - johnnynunez opened this pull request 12 days ago

[Model] Explicit interface for vLLM models and support OOT embedding models

github.com/vllm-project/vllm - DarkLight1337 opened this pull request 12 days ago

[Usage]: chat 接口有问题，completion接口正常

github.com/vllm-project/vllm - cdhx opened this issue 12 days ago

[core] remove beam search from the core

github.com/vllm-project/vllm - youkaichao opened this pull request 12 days ago

[Misc] Remove user-facing error for removed VLM args

github.com/vllm-project/vllm - DarkLight1337 opened this pull request 12 days ago

[BugFix][Core] Fix BlockManagerV2 when Encoder Input is None

github.com/vllm-project/vllm - sroy745 opened this pull request 12 days ago

[torch.compile] register blocksparse attention

github.com/vllm-project/vllm - youkaichao opened this pull request 12 days ago

[Bugfix] Fix try-catch conditions to import correct Flash Attention Backend in Draft Model

github.com/vllm-project/vllm - tjtanaa opened this pull request 12 days ago

[Bug]: Try-catch conditions are incorrect to import correct ROCm Flash Attention Backend in Draft Model

github.com/vllm-project/vllm - tjtanaa opened this issue 12 days ago

[Bug]: Llama-3.2-11B-Vision-Instruct which is an encoder-decoder model fails with BlockManager V2

github.com/vllm-project/vllm - sroy745 opened this issue 12 days ago

[RFC]: hide continuous batching complexity through forward context

github.com/vllm-project/vllm - youkaichao opened this issue 13 days ago

[core] use forward context for flash infer

github.com/vllm-project/vllm - youkaichao opened this pull request 13 days ago

[Bug]: vllm serve Exception in ASGI application

github.com/vllm-project/vllm - SpaceHunterInf opened this issue 13 days ago

[Model] Make llama3.2 support multiple and interleaved images

github.com/vllm-project/vllm - xiangxu-google opened this pull request 13 days ago

[Bug]: VLLM Model Fails on Kubernetes with "CUDA error: operation not permitted when stream is capturing"

github.com/vllm-project/vllm - CREESTL opened this issue 13 days ago

[Bugfix] limit lora init id greater than 0

github.com/vllm-project/vllm - Ssunbell opened this pull request 13 days ago

[Installation]: cannot install vllm with openvino backend

github.com/vllm-project/vllm - guanxiang opened this issue 13 days ago

[Bug]: Qwen2-VL model support

github.com/vllm-project/vllm - kulievvitaly opened this issue 13 days ago

[Model] PP support for embedding models and update docs

github.com/vllm-project/vllm - DarkLight1337 opened this pull request 13 days ago

[Hardware][CPU] Cross-attention and Encoder-Decoder models support on CPU backend

github.com/vllm-project/vllm - Isotr0py opened this pull request 13 days ago

[Doc] Update README.md with Ray summit slides

github.com/vllm-project/vllm - zhuohan123 opened this pull request 13 days ago

[Frontend] API support for beam search

github.com/vllm-project/vllm - LunrEclipse opened this pull request 13 days ago

[Bugfix] Try to handle older versions of pytorch

github.com/vllm-project/vllm - bnellnm opened this pull request 13 days ago

[Misc] Fix CI lint

github.com/vllm-project/vllm - comaniac opened this pull request 14 days ago

[Bugfix] use blockmanagerv1 for encoder-decoder

github.com/vllm-project/vllm - heheda12345 opened this pull request 14 days ago

[Bugfix] Deprecate registration of custom configs to huggingface

github.com/vllm-project/vllm - heheda12345 opened this pull request 14 days ago

[Bug]: vLLM MQLLMEngine Timeout - Json Schema

github.com/vllm-project/vllm - wrisigo opened this issue 14 days ago

[Misc] Add random seed for prefix cache benchmark

github.com/vllm-project/vllm - Imss27 opened this pull request 14 days ago

[Bug]: Lack of reproducibility across multiple runs of prefix cache benchmark

github.com/vllm-project/vllm - Imss27 opened this issue 14 days ago

Yet another Prefill-Decode separation in vllm

github.com/vllm-project/vllm - chenqianfzh opened this pull request 14 days ago

Developed the PoC of dAttention support. It will utilize the similar idea of vAttention, but it introduces a new memory layout that overcomes the waste of memory of vAttention.

github.com/vllm-project/vllm - tongping opened this pull request 14 days ago

[Misc] Improved prefix cache example

github.com/vllm-project/vllm - Imss27 opened this pull request 14 days ago

[Bug]: vllm overrides transformer's Autoconfig for mllama

github.com/vllm-project/vllm - lyuqin-scale opened this issue 14 days ago

Remove AMD Ray Summit Banner

github.com/vllm-project/vllm - simon-mo opened this pull request 14 days ago

[Doc]: Clear documentation about function / tool calling with examples

github.com/vllm-project/vllm - greg2705 opened this issue 14 days ago

[Installation]: Build failed with error : Feature 'f16 arithemetic and compare instructions' requires .target sm_53 or higher

github.com/vllm-project/vllm - ReeceResearch opened this issue 14 days ago

[Misc]: Need to understand support for torch.compile in Q4 roadmap

github.com/vllm-project/vllm - amd-abhikulk opened this issue 14 days ago

[Bugfix] Reshape the dimensions of the input image embeddings in Qwen2VL

github.com/vllm-project/vllm - whyiug opened this pull request 14 days ago

[Usage]: Benchmarking Issues: Low Success Rate and Tensor Parallel Size Constraints on 8x AMD MI300x GPUs

github.com/vllm-project/vllm - Bihan opened this issue 14 days ago

[Bug]: Issue with Pixtral Model: Unsupported Vision Configuration in vLLM ( AMD amd 7900 xtx)

github.com/vllm-project/vllm - matrix1233 opened this issue 14 days ago

[Bug]: Pixtral not working with vllm v0.6.2 docker

github.com/vllm-project/vllm - Syst3m1cAn0maly opened this issue 14 days ago

[Question]: How Does Multi-Modal LLM Handle Input Truncation Without Misaligning Image Features and Tokens?

github.com/vllm-project/vllm - HuiResearch opened this issue 14 days ago

[Doc]: Chinese Documentation Translation Available for vllm

github.com/vllm-project/vllm - khum08 opened this issue 14 days ago

[Misc] Move registry to its own file

github.com/vllm-project/vllm - DarkLight1337 opened this pull request 14 days ago

[Bug]: Lora refuses to load from disk without extremely weird manipulations with file paths

github.com/vllm-project/vllm - Ouna-the-Dataweaver opened this issue 14 days ago

[Bugfix] Flash attention arches not getting set properly

github.com/vllm-project/vllm - LucasWilkinson opened this pull request 14 days ago

[torch.compile] improve allreduce registration

github.com/vllm-project/vllm - youkaichao opened this pull request 14 days ago

[Bug]: FATAL: FlashAttention requires building with sm version sm80-sm90, but

github.com/vllm-project/vllm - 2U1 opened this issue 14 days ago

[Bug]: `"--tokenizer-mode", "mistral"` not compatible with openai API tool use tests

github.com/vllm-project/vllm - sydnash opened this issue 14 days ago

[torch.compile] integration with compilation control

github.com/vllm-project/vllm - youkaichao opened this pull request 15 days ago

[Misc] LoRA + Chunked Prefill

github.com/vllm-project/vllm - aurickq opened this pull request 15 days ago