SGLang issues | Ecosyste.ms: OpenCollective

[Eagle2]Fix multiple concurrent request crashes

github.com/sgl-project/sglang - coolhok opened this pull request 1 day ago

[Bug] add_worker API no response

github.com/sgl-project/sglang - nannaer opened this issue 1 day ago

fix end check in eagle

github.com/sgl-project/sglang - jjjjohnson opened this pull request 2 days ago

Eagle speculative decoding part 3: small modifications to the general scheduler

github.com/sgl-project/sglang - merrymercy opened this pull request 3 days ago

[Feature] Possible optimization in actor rollout parameter sync

github.com/sgl-project/sglang - 0oshowero0 opened this issue 3 days ago

Included Multinode DeepSeekv3 example

github.com/sgl-project/sglang - roG0d opened this pull request 3 days ago

[Feature] add support for deepseek v3 gptq / awq

github.com/sgl-project/sglang - Xu-Chen opened this issue 3 days ago

Improve moe reduce sum kernel performance

github.com/sgl-project/sglang - kkHuang-amd opened this pull request 3 days ago

Update documentation workflow and contribution guide

github.com/sgl-project/sglang - shuaills opened this pull request 3 days ago

[Bug] How about DeepSeek-V3 preformance with SGLang

github.com/sgl-project/sglang - liangzelang opened this issue 3 days ago

Fix CI error of nightly eval

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 3 days ago

WIP: Feature/function calling update

github.com/sgl-project/sglang - YAMY1234 opened this pull request 3 days ago

[Feature] Support regex as a stopping condition

github.com/sgl-project/sglang - mickjagger19 opened this pull request 3 days ago

[Feature] Add docs for all the deepseek model

github.com/sgl-project/sglang - zhaochenyang20 opened this issue 4 days ago

[Docs] Add Support for Structured Output Format

github.com/sgl-project/sglang - shuaills opened this pull request 4 days ago

[Feature] Add Support for Structured Output Format

github.com/sgl-project/sglang - richardodliu opened this issue 4 days ago

Speed up `update_weights_from_tensor`

github.com/sgl-project/sglang - fzyzcjy opened this pull request 4 days ago

[Feature] Reward EOS close to max_tokens

github.com/sgl-project/sglang - komninoschatzipapas opened this issue 4 days ago

Hierarchical Caching for SGLang

github.com/sgl-project/sglang - xiezhq-hermann opened this pull request 4 days ago

ROCm base image update

github.com/sgl-project/sglang - kkHuang-amd opened this pull request 4 days ago

Doc: Rename contribution_guide.md

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 5 days ago

[Docs] refactor Contribution Guide

github.com/sgl-project/sglang - shuaills opened this pull request 5 days ago

h200 tuning fused_moe_triton config for Mixtral 8x7B/8x22B and Qwen2 57BA14B

github.com/sgl-project/sglang - BBuf opened this pull request 5 days ago

Support twoshot kernel

github.com/sgl-project/sglang - yizhang2077 opened this pull request 5 days ago

[Bug] Continuous batching (OpenAI Server) with greedy search return different results

github.com/sgl-project/sglang - thangld201 opened this issue 5 days ago

[Feature] Dynamic Lora Support in SGLang (like VLLM)

github.com/sgl-project/sglang - grahama1970 opened this issue 5 days ago

[Fix] fix openai adapter

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 days ago

Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging

github.com/sgl-project/sglang - merrymercy opened this pull request 5 days ago

Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 5 days ago

feat: use CUDA 12.4 by default (for FA3)

github.com/sgl-project/sglang - zhyncs opened this pull request 5 days ago

[Feature] support ngram

github.com/sgl-project/sglang - zhyncs opened this issue 5 days ago

misc: update CODEOWNERS

github.com/sgl-project/sglang - zhyncs opened this pull request 5 days ago

minor: cleanup sgl-kernel

github.com/sgl-project/sglang - zhyncs opened this pull request 5 days ago

Eagle speculative decoding part 1: Support target model verification in the attention backend

github.com/sgl-project/sglang - merrymercy opened this pull request 5 days ago

prometheus query return no result

github.com/sgl-project/sglang - 315930399 opened this issue 5 days ago

Add cutlass submodule for sgl-kernel

github.com/sgl-project/sglang - ispobock opened this pull request 5 days ago

[Bug] The performance of v0.4.1 on AMD GPU is lower than v0.4.0

github.com/sgl-project/sglang - wyy007 opened this issue 5 days ago

Improve the computation for time_per_output_token Prometheus metrics

github.com/sgl-project/sglang - merrymercy opened this pull request 5 days ago

[Bug] DeepSeekV3 instructions don't work for multi-node H100 setup

github.com/sgl-project/sglang - mycpuorg opened this issue 5 days ago

Tiny update scripts to fail fast

github.com/sgl-project/sglang - fzyzcjy opened this pull request 5 days ago

[Bug] HuggingFace and SGLang inference don't match

github.com/sgl-project/sglang - pratcooper opened this issue 6 days ago

Minor follow-up fixes for the logprob refactor

github.com/sgl-project/sglang - merrymercy opened this pull request 6 days ago

Add GemLite caching after each capture

github.com/sgl-project/sglang - mobicham opened this pull request 6 days ago

How to obtain the hidden states of generated tokens?

github.com/sgl-project/sglang - jinhaoduan opened this issue 6 days ago

AMD DeepSeek_V3 FP8 Numerical fix

github.com/sgl-project/sglang - HaiShaw opened this pull request 6 days ago

Update structured_outputs.ipynb

github.com/sgl-project/sglang - merrymercy opened this pull request 6 days ago

Online serving benchmarks [multiturn chat, shared prefix] to multi-tier KV caching

github.com/sgl-project/sglang - PanJason opened this pull request 6 days ago

Refactor logprob computation to return the real logprob used in sampling

github.com/sgl-project/sglang - merrymercy opened this pull request 6 days ago

[feat] Add math eval to CI nightly run

github.com/sgl-project/sglang - XiaotongJiang opened this pull request 6 days ago

[Feature] Change contribution guide