github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

fix: update xgrammar v0.1.6

zhyncs opened this pull request about 2 months ago

[Feature] SGLang Router design discussion

zhyncs opened this issue about 2 months ago

Fp8 MoE optimizations on AMD

HaiShaw opened this pull request about 2 months ago

fix: resolve fp8 moe issue

zhyncs opened this pull request about 2 months ago

[Bug] circular import error in fused_moe_triton

BBuf opened this issue about 2 months ago

[Feature] Support new parameter - EBNF in xgrammar

adarshxs opened this pull request about 2 months ago

[Bug] Deepseek-v2-lite AMD MI300 run failed

BruceXcluding opened this issue about 2 months ago

Add support for Phi3V

ravi03071991 opened this pull request about 2 months ago

nit: Remove busy waiting on scheduler

rkooo567 opened this pull request about 2 months ago

Support for Pixtral model (Mistral)

yixin-huang1 opened this pull request about 2 months ago

[router] Add remove worker api

ByronHsu opened this pull request about 2 months ago

[router] add remove tenant method in the radix tree

ByronHsu opened this pull request about 2 months ago

[Router] remove duplicate char count

ByronHsu opened this pull request about 2 months ago

Fix the overlap for xgrammar

merrymercy opened this pull request about 2 months ago

[Feature] Support EBNF in xgrammar

merrymercy opened this issue about 2 months ago

Release v0.4.0.post1

merrymercy opened this pull request about 2 months ago

Use proc.join instead of busy waiting

merrymercy opened this pull request about 2 months ago

docs: update adoption (Meituan)

zhyncs opened this pull request about 2 months ago

[Feature] lora serving performance

MichoChan opened this issue about 2 months ago

MoE Expert Parallel

xiaobochen123 opened this pull request about 2 months ago

Move FP8 to SGLang

zhyncs opened this pull request about 2 months ago

[router] support `/add_worker` api

ByronHsu opened this pull request about 2 months ago

[router] use 2-gpu-runner

ByronHsu opened this pull request about 2 months ago

[Bug] sglang isn't compatible with latest vLLM

hliuca opened this issue about 2 months ago

Move FP8 to sglang

HaiShaw opened this pull request about 2 months ago

Qwen2vl vision encoder fix

jakep-allenai opened this pull request about 2 months ago

Fix AWQ with enable MLA

ispobock opened this pull request about 2 months ago

docs: Improve instructions for supporting new models

vchzls opened this pull request about 2 months ago

[Bug] deepseek v2.5 w4a16 can not run

cxmt-ai-tc opened this issue about 2 months ago

[Bug] sglang-router failure

fengyang95 opened this issue about 2 months ago

optimize cuda graph max_bs_settings on low-end gpus

BBuf opened this pull request about 2 months ago

Fix the cuda graph capture range for small #max-running-requests

merrymercy opened this pull request about 2 months ago

[Bug] runtime_endpoint encounter a JSONDecodeError

huyiwen opened this issue about 2 months ago

Add more support for intel Gaudi accelerators

YangQun1 opened this pull request about 2 months ago

[Bug] ValueError: Model architectures ['Qwen2ForCausalLM'] are not supported for now. Supported architectures: dict_keys(['CohereForCausalLM', 'DbrxForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'OlmoForCausalLM', 'Phi3SmallForCausalLM', 'StableLmForCausalLM', 'XverseForCausalLM', 'XverseMoeForCausalLM'])

Trangle opened this issue about 2 months ago

[Minor] Code style improvements

merrymercy opened this pull request about 2 months ago

[Bug] tp == 2 model gibberish

chalo2000 opened this issue about 2 months ago

Dynamo doesn't handle branching on AsyncCollectiveTensor well

bdhirsh opened this issue about 2 months ago

Make torch TP composable with torch.compile

kwen2501 opened this pull request about 2 months ago

[Feature] Support mistralai/Pixtral

merrymercy opened this issue about 2 months ago

minor: limit the range of vllm versions

zhyncs opened this pull request about 2 months ago

MLA prefill w/o weight absorption

ispobock opened this pull request about 2 months ago

Adding SGLang FP8 Utils

HaiShaw opened this pull request about 2 months ago

[Bug] Re-enable fused_moe_triton on AMD

HaiShaw opened this issue about 2 months ago

[Bug] qwen2-vl is incompatible with torch compile

wellhowtosay opened this issue about 2 months ago

[Feature] Serving VLM VILA

anhnhust opened this issue about 2 months ago

[Bug] alignment error when continuous batching and disable radix_tree_cache

kaixarider opened this issue about 2 months ago

[Feature] router adds add_worker_url, remove_worker_url api

81549361 opened this issue about 2 months ago

move apply_torchao_config_ to model_runner

jerryzh168 opened this pull request about 2 months ago

docs: add SGLang v0.4 blog

zhyncs opened this pull request about 2 months ago

Check gpu availability at server args creation

MrAta opened this pull request about 2 months ago

[router] Copy license when publishing & bump version

ByronHsu opened this pull request about 2 months ago

chore: bump v0.4.0

zhyncs opened this pull request about 2 months ago

[Feature] use SGLang's FusedMoE with quantization

zhyncs opened this issue about 2 months ago

[Feature] support AWQ with enable MLA

zhyncs opened this issue about 2 months ago

fix: resolve cmake url for Dockerfile.dev

zhyncs opened this pull request about 2 months ago

[kernel] introduce fused_moe_triton_splitk optimization

BBuf opened this pull request about 2 months ago

[kernel] introduce fused_moe_triton_splitk to sglang

BBuf opened this pull request about 2 months ago

[Feature] how to use tp or dp in offline engine?

chesterout opened this issue about 2 months ago

Fix shape error that occurred when loading lora weight of gemma2 model.

upskyy opened this pull request about 2 months ago

Revert "[feat] Enable chunked prefill for llava-onevision"

Ying1123 opened this pull request about 2 months ago

ROCm Container: set SGLANG_SET_CPU_AFFINITY=1

HaiShaw opened this pull request about 2 months ago

Improve torch compile for fused moe

merrymercy opened this pull request about 2 months ago

[Minor] Fix logger and style

merrymercy opened this pull request about 2 months ago

Add missing license for router wheel

MrAta opened this pull request about 2 months ago

Fix Docs CI When Compile Error

zhaochenyang20 opened this pull request about 2 months ago

[Bug] Update weights from disk ends in runtime corruption for different size model

zhaochenyang20 opened this issue about 2 months ago

Adapt vllm custom ar into sgl-kernel

yizhang2077 opened this pull request about 2 months ago

[Feature] Enable SGLang on more AMD GPUs

HaiShaw opened this issue about 2 months ago

Relax to include more AMD GPUs

HaiShaw opened this pull request about 2 months ago

Update model_loader deps and qqq quantization deps (#2220)

zhyncs opened this pull request about 2 months ago

[Feature] add Dockerfile dev image and doc

zhyncs opened this issue about 2 months ago

Master

ykcombat opened this pull request about 2 months ago

[Bug] fix code scanning issue

zhyncs opened this issue about 2 months ago

Add more fused moe benchmark utilities

merrymercy opened this pull request about 2 months ago

[Feature] Specify dtype at begin_forward for FlashInfer > 0.1.6

zhyncs opened this issue about 2 months ago

[Bug] Overlap mode scheduler doesn't work for bench_serving with given request rate

ykcombat opened this issue about 2 months ago

[Minor] Fix code style

merrymercy opened this pull request about 2 months ago

Use rocminfo instead of rocm-smi for more OS/WSL support

HaiShaw opened this pull request about 2 months ago

[Fix] Fix the padded hash value for image tokens

merrymercy opened this pull request about 2 months ago

Add Docs For SGLang Native Router

zhaochenyang20 opened this pull request about 2 months ago

misc: Fix typo "resulve" to "resolve"

Edenzzzz opened this pull request about 2 months ago

misc: update build setup

zhyncs opened this pull request about 2 months ago

fix: resolve CodeQL cpp issue

zhyncs opened this pull request about 2 months ago

feat: use warp reduce as a simple example

zhyncs opened this pull request about 2 months ago

[Feature] make the compilation of torch.compile faster

merrymercy opened this issue about 2 months ago

feat: support sgl-kernel pypi

zhyncs opened this pull request about 2 months ago

Fix logprob for completions

merrymercy opened this pull request about 2 months ago

Fix gptq for moe layers

merrymercy opened this pull request about 2 months ago

minor: rm unused _grouped_size_compiled_for_decode_kernels

zhyncs opened this pull request about 2 months ago

feat: skip good first issue

zhyncs opened this pull request about 2 months ago

[Feature] sgl-kernel pipelines

zhyncs opened this issue about 2 months ago

minor: support flashinfer nightly

zhyncs opened this pull request about 2 months ago

[Bug] EOFError

HuanzhiMao opened this issue about 2 months ago

[CI] Balance CI tests

merrymercy opened this pull request about 2 months ago

Feat: upgrade outlines & support compatibility with the old version

gobraves opened this pull request about 2 months ago

[Feature] Support a custom logit processor

merrymercy opened this issue about 2 months ago

Fix chunked prefill when ignore eos

hnyls2002 opened this pull request about 2 months ago

feat: add Dockerfile for development

zhyncs opened this pull request about 2 months ago

[CI] Fix missing files in run_suite.py

merrymercy opened this pull request about 2 months ago