github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

Why does the number of requests per second of 8 a800 machines not increase linearly compared to 1 a800 machine?

tensorflowt opened this issue about 1 month ago

[fix] Fix prefix caching for multi-image/video

Ying1123 opened this pull request about 1 month ago

Fix memory leak during abort

merrymercy opened this pull request about 1 month ago

[Feature] QwQ support

zhyncs opened this issue about 1 month ago

[fix] Fix multi-image/video with prefix caching

Ying1123 opened this pull request about 1 month ago

Disable overlap scheduler for multimodal models

merrymercy opened this pull request about 1 month ago

Fix chunked prefill size for bench_offline_throughput

merrymercy opened this pull request about 1 month ago

Add OLMo2 model.

janimo opened this pull request about 1 month ago

Update Install Method 2. From source

HaiShaw opened this pull request about 1 month ago

Crash the server correctly during error

merrymercy opened this pull request about 1 month ago

Lint

BBuf opened this pull request about 1 month ago

Update weights from dist [WIP]

zhaochenyang20 opened this pull request about 1 month ago

Rename tuned MI300X config files for fused_moe_triton

HaiShaw opened this pull request about 1 month ago

Bump sglang-router to 0.0.10 for env name change

ByronHsu opened this pull request about 1 month ago

[benchmark] Add fused_moe_triton benchmark and tuning tools

BBuf opened this pull request about 1 month ago

fix typo prompts

qibaoyuan opened this pull request about 1 month ago

Remove fused_moe_grok

merrymercy opened this pull request about 1 month ago

Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default

merrymercy opened this pull request about 1 month ago

Revert "Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default"

merrymercy opened this pull request about 1 month ago

Update model_loader deps and qqq quantization deps

HandH1998 opened this pull request about 1 month ago

[Feature] support torchao for qwen2 models

tricky61 opened this issue about 1 month ago

Rename DP_RANK to SGLANG_DP_RANK

merrymercy opened this pull request about 1 month ago

Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default

merrymercy opened this pull request about 1 month ago

[Bug] (v0.3.6.post2) Output degredation when using structured output

Quang-elec44 opened this issue about 1 month ago

[FEAT] Support GGUF format

zhengy001 opened this pull request about 1 month ago

Release v0.3.6.post2

merrymercy opened this pull request about 1 month ago

[Feature] ROCm in WSL environment is supported

githust66 opened this issue about 1 month ago

Fix flasky tests

merrymercy opened this pull request about 1 month ago

为啥这里是异步拷贝，然后后面直接使用了？没有显示同步？

MichoChan opened this issue about 1 month ago

[feat] Support session control for vision language models

Ying1123 opened this pull request about 1 month ago

[Bug] (Xgrammar) (v0.3.6.post1) RuntimeError: CUDA dependencies are not installed.

Quang-elec44 opened this issue about 1 month ago

Fix rust warning

ByronHsu opened this pull request about 1 month ago

Bump router to 0.0.9 with better logging

ByronHsu opened this pull request about 1 month ago

add sglang version to get_server_info

binarycrayon opened this pull request about 1 month ago

[Feature] Add sglang version to server info

binarycrayon opened this issue about 1 month ago

docs: update adoption

zhyncs opened this pull request about 1 month ago

MoE Expert Parallel Impl

xiaobochen123 opened this pull request about 1 month ago

minor: update check_env

zhyncs opened this pull request about 1 month ago

[Feature] 2:4 sparsity support

arunpatala opened this issue about 1 month ago

fix: add cuda-python for xgrammar

zhyncs opened this pull request about 1 month ago

Remove unresolved reference 'self'

apemost opened this pull request about 1 month ago

Update sampler.py to skip the success check

merrymercy opened this pull request about 1 month ago

using `is not` not `!=` to test `None`

WrRan opened this pull request about 1 month ago

remove unused imports

WrRan opened this pull request about 1 month ago

[Question] RuntimeError: Initialization failed

LiYuhang9527 opened this issue about 1 month ago

Support top n sigma sampling

Snowdar opened this pull request about 1 month ago

[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs

kkHuang-amd opened this pull request about 1 month ago

[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs.

kkHuang-amd opened this pull request about 1 month ago

Release v0.3.6.post1

merrymercy opened this pull request about 1 month ago

Rename double sparsity config file

merrymercy opened this pull request about 1 month ago

[CI] Minor fix for CI

merrymercy opened this pull request about 1 month ago

Update CI threshold

merrymercy opened this pull request about 1 month ago

Use custom allreduce w/ torch.compile

merrymercy opened this pull request about 1 month ago

[router] Rust e2e test

ByronHsu opened this pull request about 1 month ago

[router] Replace print with logger

ByronHsu opened this pull request about 1 month ago

Bump rustls from 0.23.16 to 0.23.18 in /rust

dependabot[bot] opened this pull request about 2 months ago

[Bug] Qwen2-VL-7B IndexError

jakep-allenai opened this issue about 2 months ago

[CI] Split test cases in CI for better load balancing

merrymercy opened this pull request about 2 months ago

feat: add should_use_tensor_core

zhyncs opened this pull request about 2 months ago

[Feature] Get the real logprobs to analyze decoding

Snowdar opened this issue about 2 months ago

[Bug] frequency penalty

vivian0429 opened this issue about 2 months ago

Update XGrammar to the latest API

Ubospica opened this pull request about 2 months ago

[Fix] Avoid calling fill_vocab_mask for terminated requests

Ubospica opened this pull request about 2 months ago

feat: fused_moe fp8 monkey patch

zhyncs opened this pull request about 2 months ago

[feat] Refactor session control interface and add CI

Ying1123 opened this pull request about 2 months ago

Question about ragged wrapper

ZhongYingMatrix opened this issue about 2 months ago

[Performance]: Process affinity to CPU cores with multiple sockets support

HaiShaw opened this pull request about 2 months ago

Replace prob based with threshold based load balancing

ByronHsu opened this pull request about 2 months ago

Allow overwrite flashinfer use_tensorcore

merrymercy opened this pull request about 2 months ago

[Feature] How to accelerate constrained decoding when regex needs to change with input?

GrittyChen opened this issue about 2 months ago

[Fused moe] add tuning fused configs for qwen2 57b and mixtral 8x7b

BBuf opened this pull request about 2 months ago

[Bug] cannot import name 'CachedGrammarCompiler' from 'xgrammar' (version 0.3.6)

Quang-elec44 opened this issue about 2 months ago

Test indeterminism of sgl.select under high concurrency

qeternity opened this pull request about 2 months ago

Fix docs

merrymercy opened this pull request about 2 months ago

Rename triton_fused_moe -> fused_moe_triton

merrymercy opened this pull request about 2 months ago

Balance CI tests

merrymercy opened this pull request about 2 months ago

fix: use torch.sum for compatible

zhyncs opened this pull request about 2 months ago

[Bug] FusedMoE compatible with vllm 0.6.3.post1

zhyncs opened this issue about 2 months ago

Update CI threshold & Improve code style

merrymercy opened this pull request about 2 months ago

Fix mixed chunked prefill in overlap mode

merrymercy opened this pull request about 2 months ago

fix: resolve end-of-file-fixer

zhyncs opened this pull request about 2 months ago

feat: update other MoE models deps

zhyncs opened this pull request about 2 months ago

feat: update gitignore and add tuning config for FusedMoE

zhyncs opened this pull request about 2 months ago

Simplify `Scheduler.update_running_batch`

merrymercy opened this pull request about 2 months ago

feat: remove the dependency on FusedMoE

zhyncs opened this pull request about 2 months ago

Merged three native APIs into one: get_server_info

henryhmko opened this pull request about 2 months ago

[Bug] llava use image hash as token，leading to cache bug

zwc163 opened this issue about 2 months ago

Eagle speculative decoding part 4: Add EAGLE2 worker

yukavio opened this pull request about 2 months ago

Byhsu/fairness router

ByronHsu opened this pull request about 2 months ago

Improve sglang router

ByronHsu opened this pull request about 2 months ago

add prefix match for certain tenant

ByronHsu opened this pull request about 2 months ago

Add more api routes (completion, health, etc) to the router

ByronHsu opened this pull request about 2 months ago

[Draft] Resolving integration differences after XGrammar lauch refactoring

gittb opened this pull request about 2 months ago

fix dp_rank env

ByronHsu opened this pull request about 2 months ago

update router doc

ByronHsu opened this pull request about 2 months ago

Bump sglang-router to 0.0.5

ByronHsu opened this pull request about 2 months ago

[Bug] Error when using LLAVA 1.5 for llava bench

pspdada opened this issue about 2 months ago

fix: resolve bench_serving args

zhyncs opened this pull request about 2 months ago

Fix dp print message

merrymercy opened this pull request about 2 months ago

[CI] Fix test cases

merrymercy opened this pull request about 2 months ago