github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

[Bug] alignment error when continuous batching and disable radix_tree_cache

kaixarider opened this issue about 1 month ago

[Feature] router adds add_worker_url, remove_worker_url api

81549361 opened this issue about 1 month ago

move apply_torchao_config_ to model_runner

jerryzh168 opened this pull request about 1 month ago

docs: add SGLang v0.4 blog

zhyncs opened this pull request about 1 month ago

Check gpu availability at server args creation

MrAta opened this pull request about 1 month ago

[router] Copy license when publishing & bump version

ByronHsu opened this pull request about 1 month ago

chore: bump v0.4.0

zhyncs opened this pull request about 1 month ago

[Feature] use SGLang's FusedMoE with quantization

zhyncs opened this issue about 1 month ago

[Feature] support AWQ with enable MLA

zhyncs opened this issue about 1 month ago

fix: resolve cmake url for Dockerfile.dev

zhyncs opened this pull request about 1 month ago

[kernel] introduce fused_moe_triton_splitk optimization

BBuf opened this pull request about 1 month ago

[kernel] introduce fused_moe_triton_splitk to sglang

BBuf opened this pull request about 1 month ago

[Feature] how to use tp or dp in offline engine?

chesterout opened this issue about 1 month ago

Fix shape error that occurred when loading lora weight of gemma2 model.

upskyy opened this pull request about 1 month ago

Revert "[feat] Enable chunked prefill for llava-onevision"

Ying1123 opened this pull request about 1 month ago

ROCm Container: set SGLANG_SET_CPU_AFFINITY=1

HaiShaw opened this pull request about 1 month ago

Improve torch compile for fused moe

merrymercy opened this pull request about 1 month ago

[Minor] Fix logger and style

merrymercy opened this pull request about 1 month ago

Add missing license for router wheel

MrAta opened this pull request about 1 month ago

Fix Docs CI When Compile Error

zhaochenyang20 opened this pull request about 1 month ago

[Bug] Update weights from disk ends in runtime corruption for different size model

zhaochenyang20 opened this issue about 1 month ago

Adapt vllm custom ar into sgl-kernel

yizhang2077 opened this pull request about 1 month ago

[Feature] Enable SGLang on more AMD GPUs

HaiShaw opened this issue about 1 month ago

Relax to include more AMD GPUs

HaiShaw opened this pull request about 1 month ago

Update model_loader deps and qqq quantization deps (#2220)

zhyncs opened this pull request about 1 month ago

[Feature] add Dockerfile dev image and doc

zhyncs opened this issue about 1 month ago

Master

ykcombat opened this pull request about 1 month ago

[Bug] fix code scanning issue

zhyncs opened this issue about 1 month ago

Add more fused moe benchmark utilities

merrymercy opened this pull request about 1 month ago

[Feature] Specify dtype at begin_forward for FlashInfer > 0.1.6

zhyncs opened this issue about 1 month ago

[Bug] Overlap mode scheduler doesn't work for bench_serving with given request rate

ykcombat opened this issue about 1 month ago

[Minor] Fix code style

merrymercy opened this pull request about 1 month ago

Use rocminfo instead of rocm-smi for more OS/WSL support

HaiShaw opened this pull request about 1 month ago

[Fix] Fix the padded hash value for image tokens

merrymercy opened this pull request about 1 month ago

Add Docs For SGLang Native Router

zhaochenyang20 opened this pull request about 1 month ago

misc: Fix typo "resulve" to "resolve"

Edenzzzz opened this pull request about 1 month ago

misc: update build setup

zhyncs opened this pull request about 1 month ago

fix: resolve CodeQL cpp issue

zhyncs opened this pull request about 1 month ago

feat: use warp reduce as a simple example

zhyncs opened this pull request about 1 month ago

[Feature] make the compilation of torch.compile faster

merrymercy opened this issue about 1 month ago

feat: support sgl-kernel pypi

zhyncs opened this pull request about 1 month ago

Fix logprob for completions

merrymercy opened this pull request about 1 month ago

Fix gptq for moe layers

merrymercy opened this pull request about 1 month ago

minor: rm unused _grouped_size_compiled_for_decode_kernels

zhyncs opened this pull request about 1 month ago

feat: skip good first issue

zhyncs opened this pull request about 1 month ago

[Feature] sgl-kernel pipelines

zhyncs opened this issue about 1 month ago

minor: support flashinfer nightly

zhyncs opened this pull request about 1 month ago

[Bug] EOFError

HuanzhiMao opened this issue about 1 month ago

[CI] Balance CI tests

merrymercy opened this pull request about 1 month ago

Feat: upgrade outlines & support compatibility with the old version

gobraves opened this pull request about 1 month ago

[Feature] Support a custom logit processor

merrymercy opened this issue about 1 month ago

Fix chunked prefill when ignore eos

hnyls2002 opened this pull request about 1 month ago

feat: add Dockerfile for development

zhyncs opened this pull request about 1 month ago

[CI] Fix missing files in run_suite.py

merrymercy opened this pull request about 1 month ago

Revert "Revert "[FEAT] Support GGUF format""

merrymercy opened this pull request about 1 month ago

Revert "[Fix] fix assertion error for chunked prefill when disabling cache"

merrymercy opened this pull request about 1 month ago

Revert "[FEAT] Support GGUF format"

merrymercy opened this pull request about 1 month ago

[CI] Fix ci tests

merrymercy opened this pull request about 1 month ago

[Fix] fix assertion error for chunked prefill when disabling cache

wangraying opened this pull request about 1 month ago

[feat] Enable chunked prefill for llava-onevision

Ying1123 opened this pull request about 1 month ago

[CI] Kill zombie processes

merrymercy opened this pull request about 1 month ago

Online weight updates from torch.distributed

zhaochenyang20 opened this pull request about 1 month ago

[Performance] Torch.compile is slow on MoE layers when bs > 1

merrymercy opened this issue about 1 month ago

[CI] Add accuracy test for multimodal models

merrymercy opened this issue about 1 month ago

[Feature] Support outlines >= 0.1

merrymercy opened this issue about 1 month ago

[CI] Print nightly evaluation results to GITHUB_STEP_SUMMARY

merrymercy opened this issue about 1 month ago

[CI] Print summary on github actions

merrymercy opened this pull request about 1 month ago

[Kernel] Launch two kernels for mixed chunked prefill

merrymercy opened this issue about 1 month ago

[Kernel] cuDNN attention backend

merrymercy opened this issue about 1 month ago

[Kernel] Optimize triton decoding kernels for long context

merrymercy opened this issue about 1 month ago

[Feature] support gptq or awq for deepseek v2

Xu-Chen opened this issue about 1 month ago

Add new contributors so they can trigger CI automatically

merrymercy opened this pull request about 1 month ago

Fix the default chunked prefill size

merrymercy opened this pull request about 1 month ago

update weights from distributed

zhaochenyang20 opened this pull request about 1 month ago

add get weights by parameter name for llama

zhaochenyang20 opened this pull request about 1 month ago

udate weights from disk

zhaochenyang20 opened this pull request about 1 month ago

Sgl online weights update [WIP]

zhaochenyang20 opened this pull request about 1 month ago

[Bug] Image Prompts crashing sglang server in environment without Internet (llama 3.2)

adamocarolli opened this issue about 1 month ago

Get parameter by name

zhaochenyang20 opened this pull request about 1 month ago

minor: add sgl-kernel dir

zhyncs opened this pull request about 1 month ago

rename update weights from disk api

zhaochenyang20 opened this pull request about 1 month ago

chore: bump v0.3.6.post3

zhyncs opened this pull request about 1 month ago

[Bug] flashinfer's RMSNorm implementation causes precision differences in model outputs compared to the HuggingFace implementation

BBuf opened this issue about 1 month ago

[Minor] fix the style for multimodal models

merrymercy opened this pull request about 1 month ago

Fix hash collision for multi modal models

merrymercy opened this pull request about 1 month ago

[Bug] AssertionError: assert self.being_chunked_req is None, related to `chunked_prefill_size` and input_len

HaiShaw opened this issue about 1 month ago

Simplify tokenizer manager

merrymercy opened this pull request about 1 month ago

Revert "Revert "Add simple CPU offloading support""

Ying1123 opened this pull request about 1 month ago

Revert "Add simple CPU offloading support"

Ying1123 opened this pull request about 1 month ago

Update backend.md

merrymercy opened this pull request about 1 month ago

Update backend.md

merrymercy opened this pull request about 1 month ago

[Feature] Windows OS support sglang

Zhangwt95 opened this issue about 1 month ago

Update ModelRunner Weights From Distributed

zhaochenyang20 opened this pull request about 1 month ago

Openai api supports lora path

ccchow opened this pull request about 1 month ago

[question] i wander what's the normal speed of sglang at a 8 * a800 machine.

guox18 opened this issue about 1 month ago

[Track] progress in removing vLLM dependencies

zhyncs opened this issue about 1 month ago

adapt vllm distributed module to sglang

yizhang2077 opened this pull request about 1 month ago

Support LoRA in Completion API

bjmsong opened this pull request about 1 month ago

fix missing launch server import

qeternity opened this pull request about 1 month ago

Add a simple torch native attention backend

YangQun1 opened this pull request about 1 month ago