github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

fix: resolve fp8 moe issue

zhyncs opened this pull request 16 days ago

[Bug] circular import error in fused_moe_triton

BBuf opened this issue 16 days ago

[Feature] Support new parameter - EBNF in xgrammar

adarshxs opened this pull request 16 days ago

[Bug] Deepseek-v2-lite AMD MI300 run failed

BruceXcluding opened this issue 16 days ago

Add support for Phi3V

ravi03071991 opened this pull request 16 days ago

nit: Remove busy waiting on scheduler

rkooo567 opened this pull request 16 days ago

Support for Pixtral model (Mistral)

yixin-huang1 opened this pull request 16 days ago

[router] Add remove worker api

ByronHsu opened this pull request 16 days ago

[router] add remove tenant method in the radix tree

ByronHsu opened this pull request 16 days ago

[Router] remove duplicate char count

ByronHsu opened this pull request 16 days ago

Fix the overlap for xgrammar

merrymercy opened this pull request 17 days ago

[Feature] Support EBNF in xgrammar

merrymercy opened this issue 17 days ago

Release v0.4.0.post1

merrymercy opened this pull request 17 days ago

Use proc.join instead of busy waiting

merrymercy opened this pull request 17 days ago

docs: update adoption (Meituan)

zhyncs opened this pull request 17 days ago

[Feature] lora serving performance

MichoChan opened this issue 17 days ago

MoE Expert Parallel

xiaobochen123 opened this pull request 17 days ago

Move FP8 to SGLang

zhyncs opened this pull request 17 days ago

[router] support `/add_worker` api

ByronHsu opened this pull request 17 days ago

[router] use 2-gpu-runner

ByronHsu opened this pull request 17 days ago

[Bug] sglang isn't compatible with latest vLLM

hliuca opened this issue 17 days ago

Move FP8 to sglang

HaiShaw opened this pull request 17 days ago

Qwen2vl vision encoder fix

jakep-allenai opened this pull request 17 days ago

Fix AWQ with enable MLA

ispobock opened this pull request 18 days ago

docs: Improve instructions for supporting new models

vchzls opened this pull request 18 days ago

[Bug] deepseek v2.5 w4a16 can not run

cxmt-ai-tc opened this issue 18 days ago

[Bug] sglang-router failure

fengyang95 opened this issue 18 days ago

optimize cuda graph max_bs_settings on low-end gpus

BBuf opened this pull request 18 days ago

Fix the cuda graph capture range for small #max-running-requests

merrymercy opened this pull request 18 days ago

[Bug] runtime_endpoint encounter a JSONDecodeError

huyiwen opened this issue 18 days ago

Add more support for intel Gaudi accelerators

YangQun1 opened this pull request 18 days ago

[Bug] ValueError: Model architectures ['Qwen2ForCausalLM'] are not supported for now. Supported architectures: dict_keys(['CohereForCausalLM', 'DbrxForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'OlmoForCausalLM', 'Phi3SmallForCausalLM', 'StableLmForCausalLM', 'XverseForCausalLM', 'XverseMoeForCausalLM'])

Trangle opened this issue 18 days ago

[Minor] Code style improvements

merrymercy opened this pull request 18 days ago

[Bug] tp == 2 model gibberish

chalo2000 opened this issue 18 days ago

Dynamo doesn't handle branching on AsyncCollectiveTensor well

bdhirsh opened this issue 18 days ago

Make torch TP composable with torch.compile

kwen2501 opened this pull request 18 days ago

[Feature] Support mistralai/Pixtral

merrymercy opened this issue 18 days ago

minor: limit the range of vllm versions

zhyncs opened this pull request 19 days ago

MLA prefill w/o weight absorption

ispobock opened this pull request 19 days ago

Adding SGLang FP8 Utils

HaiShaw opened this pull request 19 days ago

[Bug] Re-enable fused_moe_triton on AMD

HaiShaw opened this issue 19 days ago

[Bug] qwen2-vl is incompatible with torch compile

wellhowtosay opened this issue 19 days ago

[Feature] Serving VLM VILA

anhnhust opened this issue 19 days ago

[Bug] alignment error when continuous batching and disable radix_tree_cache

kaixarider opened this issue 19 days ago

[Feature] router adds add_worker_url, remove_worker_url api

81549361 opened this issue 19 days ago

move apply_torchao_config_ to model_runner

jerryzh168 opened this pull request 19 days ago

docs: add SGLang v0.4 blog

zhyncs opened this pull request 19 days ago

Check gpu availability at server args creation

MrAta opened this pull request 19 days ago

[router] Copy license when publishing & bump version

ByronHsu opened this pull request 20 days ago

chore: bump v0.4.0

zhyncs opened this pull request 20 days ago

[Feature] use SGLang's FusedMoE with quantization

zhyncs opened this issue 20 days ago

[Feature] support AWQ with enable MLA

zhyncs opened this issue 20 days ago

fix: resolve cmake url for Dockerfile.dev

zhyncs opened this pull request 20 days ago

[kernel] introduce fused_moe_triton_splitk optimization

BBuf opened this pull request 20 days ago

[kernel] introduce fused_moe_triton_splitk to sglang

BBuf opened this pull request 20 days ago

[Feature] how to use tp or dp in offline engine?

chesterout opened this issue 20 days ago

Fix shape error that occurred when loading lora weight of gemma2 model.

upskyy opened this pull request 20 days ago

Revert "[feat] Enable chunked prefill for llava-onevision"

Ying1123 opened this pull request 20 days ago

ROCm Container: set SGLANG_SET_CPU_AFFINITY=1

HaiShaw opened this pull request 20 days ago

Improve torch compile for fused moe

merrymercy opened this pull request 20 days ago

[Minor] Fix logger and style

merrymercy opened this pull request 20 days ago

Add missing license for router wheel

MrAta opened this pull request 20 days ago

Fix Docs CI When Compile Error

zhaochenyang20 opened this pull request 20 days ago

[Bug] Update weights from disk ends in runtime corruption for different size model

zhaochenyang20 opened this issue 20 days ago

Adapt vllm custom ar into sgl-kernel

yizhang2077 opened this pull request 21 days ago

[Feature] Enable SGLang on more AMD GPUs

HaiShaw opened this issue 21 days ago

Relax to include more AMD GPUs

HaiShaw opened this pull request 21 days ago

Update model_loader deps and qqq quantization deps (#2220)

zhyncs opened this pull request 21 days ago

[Feature] add Dockerfile dev image and doc

zhyncs opened this issue 21 days ago

Master

ykcombat opened this pull request 21 days ago

[Bug] fix code scanning issue

zhyncs opened this issue 21 days ago

Add more fused moe benchmark utilities

merrymercy opened this pull request 21 days ago

[Feature] Specify dtype at begin_forward for FlashInfer > 0.1.6

zhyncs opened this issue 21 days ago

[Bug] Overlap mode scheduler doesn't work for bench_serving with given request rate

ykcombat opened this issue 21 days ago

[Minor] Fix code style

merrymercy opened this pull request 21 days ago

Use rocminfo instead of rocm-smi for more OS/WSL support

HaiShaw opened this pull request 21 days ago

[Fix] Fix the padded hash value for image tokens

merrymercy opened this pull request 21 days ago

Add Docs For SGLang Native Router

zhaochenyang20 opened this pull request 21 days ago

misc: Fix typo "resulve" to "resolve"

Edenzzzz opened this pull request 21 days ago

misc: update build setup

zhyncs opened this pull request 22 days ago

fix: resolve CodeQL cpp issue

zhyncs opened this pull request 22 days ago

feat: use warp reduce as a simple example

zhyncs opened this pull request 22 days ago

[Feature] make the compilation of torch.compile faster

merrymercy opened this issue 22 days ago

feat: support sgl-kernel pypi

zhyncs opened this pull request 22 days ago

Fix logprob for completions

merrymercy opened this pull request 22 days ago

Fix gptq for moe layers

merrymercy opened this pull request 22 days ago

minor: rm unused _grouped_size_compiled_for_decode_kernels

zhyncs opened this pull request 22 days ago

feat: skip good first issue

zhyncs opened this pull request 22 days ago

[Feature] sgl-kernel pipelines

zhyncs opened this issue 22 days ago

minor: support flashinfer nightly

zhyncs opened this pull request 22 days ago

[Bug] EOFError

HuanzhiMao opened this issue 22 days ago

[CI] Balance CI tests

merrymercy opened this pull request 22 days ago

Feat: upgrade outlines & support compatibility with the old version

gobraves opened this pull request 22 days ago

[Feature] Support a custom logit processor

merrymercy opened this issue 22 days ago

Fix chunked prefill when ignore eos

hnyls2002 opened this pull request 22 days ago

feat: add Dockerfile for development

zhyncs opened this pull request 22 days ago

[CI] Fix missing files in run_suite.py

merrymercy opened this pull request 22 days ago

Revert "Revert "[FEAT] Support GGUF format""

merrymercy opened this pull request 22 days ago

Revert "[Fix] fix assertion error for chunked prefill when disabling cache"

merrymercy opened this pull request 22 days ago

Revert "[FEAT] Support GGUF format"

merrymercy opened this pull request 22 days ago