github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

refine moe_align_kernel and fused_moe

BBuf opened this pull request 4 days ago

Support cutlass Int8 gemm

ispobock opened this pull request 4 days ago

Remove unused var in moe_align_kernel

ispobock opened this pull request 4 days ago

Fix sgl-kernel cu118 compile issue

ispobock opened this pull request 4 days ago

Allow multi SGLang engines to coordinate

fzyzcjy opened this pull request 4 days ago

Support llamafy/Qwen-Qwen2.5-7B-Instruct-llamafied

Xu-Chen opened this pull request 4 days ago

[WIP] Refactor

fzyzcjy opened this pull request 5 days ago

[Bug] Long output and issues when running benchmark_serving.py on DeepSeek-V3

lhl opened this issue 5 days ago

feat: add devcontainer.json for VSCode development

observerw opened this pull request 5 days ago

[Feature] Docs: Collect all the commands for DeepSeek in SGlang

zhaochenyang20 opened this issue 5 days ago

[Feature] Rewrite docs for LLama 405B and ModelSpace

zhaochenyang20 opened this issue 5 days ago

Update doc for server arguments

simveit opened this pull request 5 days ago

[Bug] SGLang stops working after a few requests when serving DeepSeek V3

seungduk-yanolja opened this issue 5 days ago

[Feature] support ep for DeepSeek V3

zhyncs opened this issue 5 days ago

[Feature] adapt fused sigmoid gate for MoE model

zhyncs opened this issue 5 days ago

Benchmark results for DeepSeek-v3 in 2x8xH200 Cluster

roG0d opened this pull request 5 days ago

[Feature] several features for veRL integration

PeterSH6 opened this issue 5 days ago

improve moe_align_kernel for deepseek v3

BBuf opened this pull request 5 days ago

I wonder if the offline engine API supports OpenAI input format.

hljjjmssyh opened this issue 6 days ago

fix lint

zhyncs opened this pull request 6 days ago

[Feature] optimize moe_align_block_size_kernel

zhyncs opened this issue 6 days ago

Revert the GLOO_SOCKET_IFNAME change

merrymercy opened this pull request 6 days ago

[Eagle2]Fix multiple concurrent request crashes

coolhok opened this pull request 7 days ago

[Feature] Support bitsandbytes in QWen2 VL

gaocegege opened this issue 7 days ago

[Bug] add_worker API no response

nannaer opened this issue 7 days ago

[Docs] fix 404 - Contributor Guide, again

gaocegege opened this pull request 7 days ago

feat: Support VLM in reference_hf

gaocegege opened this pull request 7 days ago

for test

zhyncs opened this pull request 8 days ago

fix end check in eagle

jjjjohnson opened this pull request 8 days ago

Update README.md

merrymercy opened this pull request 8 days ago

[Bug] How to load weight with torchao

sitabulaixizawaluduo opened this issue 8 days ago

[willing to PR] optimzation of eagle2

jjjjohnson opened this issue 8 days ago

[Bug] How to install Cuda11.8 sglang?

Meta-YZ opened this issue 8 days ago

Fix package loss for small models

merrymercy opened this pull request 8 days ago

Support loading pre-sharded moe weights

merrymercy opened this pull request 8 days ago

test: add test_block_fp8 in CI

zhyncs opened this pull request 8 days ago

[Fix] fix incorrectly overwriting the port specified in ServerArgs

mickqian opened this pull request 8 days ago

chore: bump v0.4.1.post4

zhyncs opened this pull request 8 days ago

feat: support moe_align_block_size_triton

zhyncs opened this pull request 8 days ago

[Fix] fix retract error in eagle speculative decoding

yukavio opened this pull request 8 days ago

Eagle speculative decoding part 3: small modifications to the general scheduler

merrymercy opened this pull request 9 days ago

[Feature] Possible optimization in actor rollout parameter sync

0oshowero0 opened this issue 9 days ago

Included multi-node DeepSeekv3 example

roG0d opened this pull request 9 days ago

[Feature] add support for deepseek v3 gptq / awq

Xu-Chen opened this issue 9 days ago

Improve moe reduce sum kernel performance

kkHuang-amd opened this pull request 9 days ago

Update documentation workflow and contribution guide

shuaills opened this pull request 9 days ago

[Bug] How about DeepSeek-V3 preformance with SGLang

liangzelang opened this issue 9 days ago

Fix CI error of nightly eval

zhaochenyang20 opened this pull request 9 days ago

WIP: Feature/function calling update

YAMY1234 opened this pull request 9 days ago

[Feature] Support regex as a stopping condition

mickqian opened this pull request 9 days ago

[Feature] Add docs for all the deepseek model

zhaochenyang20 opened this issue 9 days ago

[Docs] Add Support for Structured Output Format

shuaills opened this pull request 9 days ago

[Feature] Add Support for Structured Output Format

richardodliu opened this issue 9 days ago

Speed up `update_weights_from_tensor`

fzyzcjy opened this pull request 9 days ago

[Feature] Reward EOS close to max_tokens

komninoschatzipapas opened this issue 9 days ago

Hierarchical Caching for SGLang

xiezhq-hermann opened this pull request 10 days ago

ROCm base image update

kkHuang-amd opened this pull request 10 days ago

Doc: Rename contribution_guide.md

zhaochenyang20 opened this pull request 10 days ago

[Docs] refactor Contribution Guide

shuaills opened this pull request 10 days ago

h200 tuning fused_moe_triton config for Mixtral 8x7B/8x22B and Qwen2 57BA14B

BBuf opened this pull request 10 days ago

Support twoshot kernel

yizhang2077 opened this pull request 10 days ago

[Bug] Continuous batching (OpenAI Server) with greedy search return different results

thangld201 opened this issue 10 days ago

[Feature] Dynamic Lora Support in SGLang (like VLLM)

grahama1970 opened this issue 10 days ago

[Fix] fix openai adapter

Ying1123 opened this pull request 10 days ago

Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging

merrymercy opened this pull request 10 days ago

Update README.md

merrymercy opened this pull request 11 days ago

feat: use CUDA 12.4 by default (for FA3)

zhyncs opened this pull request 11 days ago

[Feature] support ngram

zhyncs opened this issue 11 days ago

misc: update CODEOWNERS

zhyncs opened this pull request 11 days ago

minor: cleanup sgl-kernel

zhyncs opened this pull request 11 days ago

Eagle speculative decoding part 1: Support target model verification in the attention backend

merrymercy opened this pull request 11 days ago

prometheus query return no result

315930399 opened this issue 11 days ago

Add cutlass submodule for sgl-kernel

ispobock opened this pull request 11 days ago

[Bug] The performance of v0.4.1 on AMD GPU is lower than v0.4.0

wyy007 opened this issue 11 days ago

Improve the computation for time_per_output_token Prometheus metrics

merrymercy opened this pull request 11 days ago

[Bug] DeepSeekV3 instructions don't work for multi-node H100 setup

mycpuorg opened this issue 11 days ago

Tiny update scripts to fail fast

fzyzcjy opened this pull request 11 days ago

[Bug] HuggingFace and SGLang inference don't match

pratcooper opened this issue 11 days ago

Minor follow-up fixes for the logprob refactor

merrymercy opened this pull request 11 days ago

Add GemLite caching after each capture

mobicham opened this pull request 11 days ago

How to obtain the hidden states of generated tokens?

jinhaoduan opened this issue 11 days ago

AMD DeepSeek_V3 FP8 Numerical fix

HaiShaw opened this pull request 11 days ago

Update structured_outputs.ipynb

merrymercy opened this pull request 12 days ago

Online serving benchmarks [multiturn chat, shared prefix] to multi-tier KV caching

PanJason opened this pull request 12 days ago

Refactor logprob computation to return the real logprob used in sampling

merrymercy opened this pull request 12 days ago

[feat] Add math eval to CI nightly run

XiaotongJiang opened this pull request 12 days ago

[Feature] Change contribution guide