SGLang issues | Ecosyste.ms: OpenCollective

Fix correctness issue for triton decoding kernel

github.com/sgl-project/sglang - ispobock opened this pull request 20 days ago

[Experimental] Add a gRPC server for completion request

github.com/sgl-project/sglang - MrAta opened this pull request 21 days ago

How to debug sglang using pdb?

github.com/sgl-project/sglang - sleepwalker2017 opened this issue 21 days ago

Small fixes for torchao quant

github.com/sgl-project/sglang - jerryzh168 opened this pull request 21 days ago

[FIX] Update EOS from config

github.com/sgl-project/sglang - zhengy001 opened this pull request 21 days ago

[Feature] request smoothquant (int8, W8A8) quantization on 40G A100

github.com/sgl-project/sglang - Hao-YunDeng opened this issue 22 days ago

[Minor] Fix grok model loader

github.com/sgl-project/sglang - merrymercy opened this pull request 22 days ago

[Feature] Integrate CUTLASS FP8 GEMM into sgl-kernel

github.com/sgl-project/sglang - zhyncs opened this issue 22 days ago

[Feature] FusedMoE H200 tuning

github.com/sgl-project/sglang - zhyncs opened this issue 22 days ago

[Bug] Different behavior benchmarking w/ request-range-range vs. separate request-rates

github.com/sgl-project/sglang - Mutinifni opened this issue 22 days ago

feat: support dev image

github.com/sgl-project/sglang - zhyncs opened this pull request 22 days ago

"GET / HTTP/1.1" 404 Not Found

github.com/sgl-project/sglang - LordEdison opened this issue 22 days ago

benchmark decoding attention kernel with cudnn

github.com/sgl-project/sglang - bjmsong opened this pull request 22 days ago

fix: set runtime path

github.com/sgl-project/sglang - zhyncs opened this pull request 22 days ago

[Bug] potential correctness with triton-attention-num-kv-splits > 1

github.com/sgl-project/sglang - HaiShaw opened this issue 22 days ago

Rename rust folder to sgl-router

github.com/sgl-project/sglang - MrAta opened this pull request 22 days ago

minor: update pypi tag

github.com/sgl-project/sglang - zhyncs opened this pull request 22 days ago

chore: bump v0.0.2 for sgl-kernel

github.com/sgl-project/sglang - zhyncs opened this pull request 22 days ago

[Feature] Do we have any plan for supporting MiniCPM-V 2.6?

github.com/sgl-project/sglang - Xeladoes opened this issue 22 days ago

[Bug] CUDA Graph Build Failure

github.com/sgl-project/sglang - dangxingyu opened this issue 22 days ago

Bump sglang-router to 0.1.1

github.com/sgl-project/sglang - MrAta opened this pull request 22 days ago

[Feature] MoE Expert Parallel with awq

github.com/sgl-project/sglang - Xu-Chen opened this issue 23 days ago

Clean up GPU memory after killing sglang processes

github.com/sgl-project/sglang - MrAta opened this pull request 23 days ago

Include version info into the router package

github.com/sgl-project/sglang - MrAta opened this pull request 23 days ago

[router] Release router 0.1.0 with dynamic scaling and fault tolerance

github.com/sgl-project/sglang - ByronHsu opened this pull request 23 days ago

[router] Update doc for dynamic scaling and fault tolerance

github.com/sgl-project/sglang - ByronHsu opened this pull request 23 days ago

[router] remove main.rs because only lib.rs is used for py binding

github.com/sgl-project/sglang - ByronHsu opened this pull request 23 days ago

[router] Add retries based fault tolerance

github.com/sgl-project/sglang - ByronHsu opened this pull request 23 days ago

[Bug] Gemma 2 GGUF

github.com/sgl-project/sglang - slivka83 opened this issue 23 days ago

[Feature]: Benchmarking H200

github.com/sgl-project/sglang - antferdom opened this issue 23 days ago

Fix warmup in bench_offline_throughput.py

github.com/sgl-project/sglang - merrymercy opened this pull request 23 days ago

Fix model loader for more quantization formats

github.com/sgl-project/sglang - merrymercy opened this pull request 23 days ago

chore: update ao v0.7.0

github.com/sgl-project/sglang - zhyncs opened this pull request 23 days ago

It's hard to install it

github.com/sgl-project/sglang - ToSev7en opened this issue 23 days ago

Make request payload size configurable

github.com/sgl-project/sglang - MrAta opened this pull request 23 days ago

[Feature] support Llama3.3

github.com/sgl-project/sglang - win4r opened this issue 23 days ago

[Core] in batch prefix caching by delay scheduling

github.com/sgl-project/sglang - rkooo567 opened this pull request 23 days ago

[router] Use borrow if possible to save cost

github.com/sgl-project/sglang - ByronHsu opened this pull request 23 days ago

[router] Refactor: decouple select and send stage

github.com/sgl-project/sglang - ByronHsu opened this pull request 23 days ago

[Feature] Enhanced support/structure for Multi-modal models

github.com/sgl-project/sglang - tp-nan opened this issue 23 days ago

Add lora_path to chat completion

github.com/sgl-project/sglang - ccchow opened this pull request 23 days ago

Add support for IBM Granite 3.x models

github.com/sgl-project/sglang - frreiss opened this pull request 24 days ago

Make torch TP composable with torchao

github.com/sgl-project/sglang - kwen2501 opened this pull request 24 days ago

fix: compatible with PEP 440

github.com/sgl-project/sglang - zhyncs opened this pull request 24 days ago

fix: use manylinux2014_x86_64 tag

github.com/sgl-project/sglang - zhyncs opened this pull request 24 days ago

feat: support sgl-kernel PyPI

github.com/sgl-project/sglang - zhyncs opened this pull request 24 days ago

[Bug] vLLM ~0.6.5 with latest sglang producing garbage text on AMD GPUs

github.com/sgl-project/sglang - ozziemoreno opened this issue 24 days ago

[Bug] SGLang's OpenAI interface fails with Llama-3.2-1B due to missing chat template

github.com/sgl-project/sglang - NeilJohnson0930 opened this issue 24 days ago

[Bug] multiple `sgl.Runtime` instances compete for port 10000

github.com/sgl-project/sglang - mantle2048 opened this issue 24 days ago

[Feature] Function Call Support

github.com/sgl-project/sglang - chenweize1998 opened this issue 24 days ago

Best practices for deploying different models on different GPUs for offline generation

github.com/sgl-project/sglang - mantle2048 opened this issue 24 days ago

[Feature] Support General Reward Model

github.com/sgl-project/sglang - zhaochenyang20 opened this issue 24 days ago

ROCm support for sglang.check_env

github.com/sgl-project/sglang - hliuca opened this pull request 25 days ago

decoding attention kernel benchmark

github.com/sgl-project/sglang - bjmsong opened this pull request 25 days ago

Typo fix in router.md

github.com/sgl-project/sglang - adarshxs opened this pull request 25 days ago

Performance issues when scaling to multiple GPUs

github.com/sgl-project/sglang - FinnGu opened this issue 25 days ago

[Minor] Improve code style

github.com/sgl-project/sglang - merrymercy opened this pull request 25 days ago

Add InfiniteBench for long context benchmarking

github.com/sgl-project/sglang - iankur opened this pull request 25 days ago

[Bug] The first request with "regex" is too slow

github.com/sgl-project/sglang - sitabulaixizawaluduo opened this issue 25 days ago

[Minor] Improve code style

github.com/sgl-project/sglang - merrymercy opened this pull request 25 days ago

[Bug] File "/u02/liuys/sglang/python/sglang/srt/server.py", line 621, in _wait_and_warmup Killed

github.com/sgl-project/sglang - lys791227 opened this issue 25 days ago

Migrate llama_classification to use the /classify interface

github.com/sgl-project/sglang - merrymercy opened this pull request 25 days ago

Add a unittest for fused_moe

github.com/sgl-project/sglang - BBuf opened this pull request 25 days ago

[Bug] nsys will cause an error when TP=4 although I launched with --trace-fork-before-exec=true --cuda-graph-trace=node

github.com/sgl-project/sglang - jameswu2014 opened this issue 25 days ago

[Bug] XGrammar causes gibberish during parallel execution and cuts off other requests

github.com/sgl-project/sglang - remixer-dec opened this issue 25 days ago

[Router] fix interrupt from terminal

github.com/sgl-project/sglang - ByronHsu opened this pull request 26 days ago

[feat] Enable chunked prefill for llava-onevision

github.com/sgl-project/sglang - Ying1123 opened this pull request 26 days ago

[router] Improve cleanup logic

github.com/sgl-project/sglang - ByronHsu opened this pull request 26 days ago

reduce watchdog interval to 5s

github.com/sgl-project/sglang - ByronHsu opened this pull request 26 days ago

minor: add random flashinfer vs triton use case

github.com/sgl-project/sglang - zhyncs opened this pull request 26 days ago

minor: add random use case

github.com/sgl-project/sglang - zhyncs opened this pull request 26 days ago

feat: support custom task runner

github.com/sgl-project/sglang - zhyncs opened this pull request 26 days ago

minor: update correct measurement unit

github.com/sgl-project/sglang - zhyncs opened this pull request 26 days ago

Fix recv_requests

github.com/sgl-project/sglang - merrymercy opened this pull request 26 days ago

fix: specify dtype with begin_forward aka plan

github.com/sgl-project/sglang - zhyncs opened this pull request 26 days ago

Fix a bug with logprob streaming + chunked prefill

github.com/sgl-project/sglang - merrymercy opened this pull request 26 days ago

[Feature] add kernel level benchmark

github.com/sgl-project/sglang - zhyncs opened this issue 26 days ago

Remove unused vars in the triton backend

github.com/sgl-project/sglang - ispobock opened this pull request 26 days ago

[Feature] support llm_bench

github.com/sgl-project/sglang - zhyncs opened this issue 26 days ago

[Feature] support constrained decoding benchmark

github.com/sgl-project/sglang - zhyncs opened this issue 26 days ago

Simplify stream_output

github.com/sgl-project/sglang - merrymercy opened this pull request 26 days ago

Update killall_sglang.sh

github.com/sgl-project/sglang - merrymercy opened this pull request 26 days ago

[WIP] Add sampler logit processor

github.com/sgl-project/sglang - hongpeng-guo opened this pull request 26 days ago

[Bug] After deploying for a period of time (2 days), the speed slows down and the memory usage increases

github.com/sgl-project/sglang - lss15151161 opened this issue 26 days ago

Optimize Triton decoding kernel for long context

github.com/sgl-project/sglang - ispobock opened this pull request 26 days ago

[router] defer health checking to router init

github.com/sgl-project/sglang - ByronHsu opened this pull request 27 days ago

[router] Health check on worker before added to the router

github.com/sgl-project/sglang - ByronHsu opened this pull request 27 days ago

minor: update killall script

github.com/sgl-project/sglang - zhyncs opened this pull request 27 days ago

fix: update xgrammar v0.1.6

github.com/sgl-project/sglang - zhyncs opened this pull request 27 days ago

[Feature] SGLang Router design discussion

github.com/sgl-project/sglang - zhyncs opened this issue 27 days ago

Fp8 MoE optimizations on AMD

github.com/sgl-project/sglang - HaiShaw opened this pull request 27 days ago

fix: resolve fp8 moe issue

github.com/sgl-project/sglang - zhyncs opened this pull request 27 days ago

[Bug] circular import error in fused_moe_triton

github.com/sgl-project/sglang - BBuf opened this issue 27 days ago

[Feature] Support new parameter - EBNF in xgrammar

github.com/sgl-project/sglang - adarshxs opened this pull request 27 days ago

[Bug] Deepseek-v2-lite AMD MI300 run failed

github.com/sgl-project/sglang - BruceXcluding opened this issue 27 days ago

Add support for Phi3V

github.com/sgl-project/sglang - ravi03071991 opened this pull request 27 days ago

nit: Remove busy waiting on scheduler

github.com/sgl-project/sglang - rkooo567 opened this pull request 27 days ago

Support for Pixtral model (Mistral)

github.com/sgl-project/sglang - yixin-huang1 opened this pull request 27 days ago

[router] Add remove worker api

github.com/sgl-project/sglang - ByronHsu opened this pull request 28 days ago

[router] add remove tenant method in the radix tree

github.com/sgl-project/sglang - ByronHsu opened this pull request 28 days ago