SGLang issues | Ecosyste.ms: OpenCollective

update pr-test ci

github.com/sgl-project/sglang - zhyncs opened this pull request about 18 hours ago

[Bug] sqlite3.DatabaseError: database disk image is malformed when deploying Deepseek-R1-bf16 with SLURM

github.com/sgl-project/sglang - lll2343 opened this issue about 18 hours ago

update sgl-kernel version

github.com/sgl-project/sglang - zhyncs opened this pull request about 18 hours ago

support speculative decoding kernel in sgl-kernel

github.com/sgl-project/sglang - zhyncs opened this pull request about 19 hours ago

fix undefined symbol cudaGetDriverEntryPointByVersion

github.com/sgl-project/sglang - zhyncs opened this pull request about 19 hours ago

[Bug] Error when Load DeepSeek-R1 Model in --enable-ep-moe

github.com/sgl-project/sglang - mengsoso opened this issue about 20 hours ago

[Bug] Hardcoded block size blocks expert parallelism on NVIDIA L40S

github.com/sgl-project/sglang - yubingjiaocn opened this issue about 20 hours ago

chore: bump v0.4.2.post3

github.com/sgl-project/sglang - zhyncs opened this pull request about 21 hours ago

[Bug] Watchdog caught collective operation timeout:

github.com/sgl-project/sglang - CallmeZhangChenchen opened this issue about 21 hours ago

Better support of tp checkpoint loading

github.com/sgl-project/sglang - yinghai opened this pull request about 23 hours ago

update unit test in AMD CI

github.com/sgl-project/sglang - zhyncs opened this pull request about 23 hours ago

[Feature] support `gather` instead of `all_gather` when gathering the logits

github.com/sgl-project/sglang - chunyuan-w opened this issue about 23 hours ago

[Feat] return hidden states

github.com/sgl-project/sglang - Jackmin801 opened this pull request 1 day ago

Add support for OpenAI API o1 model

github.com/sgl-project/sglang - ChuyueSun opened this pull request 1 day ago

run eagle speculative decodeing error!

github.com/sgl-project/sglang - v-lmn opened this issue 1 day ago

Query on Optimizing sglang Best Performance During DeepSeek V3 Inference

github.com/sgl-project/sglang - aooxin opened this issue 1 day ago

[Bug] DeepSeek-R1 NCCL WatchDog Timeout Error

github.com/sgl-project/sglang - sitabulaixizawaluduo opened this issue 1 day ago

If dp_size = tp_size is still required for deepseek model?

github.com/sgl-project/sglang - luzengxiangcn opened this issue 1 day ago

[Bug] request timeout with multi-gpu model

github.com/sgl-project/sglang - dwq370 opened this issue 1 day ago

Efficient topk

github.com/sgl-project/sglang - Jackmin801 opened this pull request 1 day ago

update waves_per_eu to 1

github.com/sgl-project/sglang - lizamd opened this pull request 1 day ago

Feature: Fix the binding error in Llama

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 1 day ago

chore: bump v0.4.2.post3

github.com/sgl-project/sglang - HaiShaw opened this pull request 1 day ago

fix(mig): fallback gpu_memory_total value

github.com/sgl-project/sglang - tomheno opened this pull request 1 day ago

fix sgl-kernel build failure on AMD

github.com/sgl-project/sglang - zhyncs opened this pull request 1 day ago

feature: Intern vl 2.5

github.com/sgl-project/sglang - mickqian opened this pull request 1 day ago

enable fake finish for docs PR

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 1 day ago

[Doc] Add optimization option guide for deepseek v3

github.com/sgl-project/sglang - ispobock opened this pull request 1 day ago

Tiny fix Olmo2

github.com/sgl-project/sglang - fzyzcjy opened this pull request 1 day ago

optimize moe_align_kernel cuda

github.com/sgl-project/sglang - BBuf opened this pull request 1 day ago

Update fused_moe's benchmark

github.com/sgl-project/sglang - WhatGhost opened this pull request 1 day ago

[Bug] deepseek-R1 671b can not set tensor_parallel_size=32

github.com/sgl-project/sglang - Anlysqx opened this issue 2 days ago

Using 2 H100*8 machines to deploy deepseek-R1 671 B, can the context length consistently support up to 128k? The GPU memory seems stable during sglang inference, but I would like to know if there is a possibility of running out of memory.

github.com/sgl-project/sglang - Anlysqx opened this issue 2 days ago

[Bug] Security Guidelines for Responsible Disclosure

github.com/sgl-project/sglang - avioligo opened this issue 2 days ago

Connection Error When Calling DeepSeek via Slurm Cluster

github.com/sgl-project/sglang - lll2343 opened this issue 2 days ago

用 2台 H100*8 部署 deepseek-R1 671 B，context length 可以一直支持到 128k吗，sglang 推理时显存貌似是稳定的，想请教会不会爆显存

github.com/sgl-project/sglang - Anlysqx opened this issue 2 days ago

[Bug] Error running Sglang with DeepSeek v3 using 4 slurm cluster nodes

github.com/sgl-project/sglang - vabatista opened this issue 2 days ago

[Bug] how to solve illegal memory access in moe_align_block_size kernel optimization

github.com/sgl-project/sglang - BBuf opened this issue 2 days ago

add AMD guide for DeepSeek-R1

github.com/sgl-project/sglang - zhyncs opened this pull request 2 days ago

update pull request template

github.com/sgl-project/sglang - zhyncs opened this pull request 2 days ago

Add sgl-kernel to MI300 CI paths tested.

github.com/sgl-project/sglang - saienduri opened this pull request 2 days ago

[Bug] DeepSeek-V3 an illegal memory access was encountered

github.com/sgl-project/sglang - seungrokj opened this issue 2 days ago

[Bug] DeepSeek-V3 an illegal memory access was encountered

github.com/sgl-project/sglang - seungrokj opened this issue 2 days ago

clean moe align block kernel code and add acc test

github.com/sgl-project/sglang - BBuf opened this pull request 2 days ago

[Bug] get_nvgpu_memory_capacity() causes crash on orin

github.com/sgl-project/sglang - ryanmafrty opened this issue 2 days ago

[Feature] Rewrite the docs of Frontend

github.com/sgl-project/sglang - zhaochenyang20 opened this issue 2 days ago

Unable to Upload Images During Inference with DeepSeek

github.com/sgl-project/sglang - zhaohm14 opened this issue 2 days ago

[Feature] grafana dashboard should work out of the box

github.com/sgl-project/sglang - ziliangpeng opened this issue 2 days ago

Docker switch on mi300 CI.

github.com/sgl-project/sglang - saienduri opened this pull request 2 days ago

[WIP] Enable DeepSeek Models on Intel Gaudi device

github.com/sgl-project/sglang - YangQun1 opened this pull request 2 days ago

[ROCm] Fix fp8 unrolledx4 matmul kernel.

github.com/sgl-project/sglang - whchung opened this pull request 2 days ago

update README

github.com/sgl-project/sglang - zhyncs opened this pull request 2 days ago

[Feature] optimize group gemm

github.com/sgl-project/sglang - zhyncs opened this issue 2 days ago

[Bug] Qwen 2.5 VL

github.com/sgl-project/sglang - grahama1970 opened this issue 2 days ago

How to load a llama model with tp sharded checkpoint?

github.com/sgl-project/sglang - sshleifer opened this issue 2 days ago

[Bug] sglang seemingly not respecting dsv3 quantization on rocm

github.com/sgl-project/sglang - AdjectiveAllison opened this issue 3 days ago

[Bug] router: prefix cache routing is not working due to mismatch in route parameter check

github.com/sgl-project/sglang - ltalal opened this issue 3 days ago

[Bug] Error when running Qwen2 EAGLE speculative decoding refering to the official example

github.com/sgl-project/sglang - feifeibear opened this issue 3 days ago

Feature/docs deepseek usage and add multi-node

github.com/sgl-project/sglang - lycanlancelot opened this pull request 3 days ago

chore: bump v0.4.2.post2

github.com/sgl-project/sglang - zhyncs opened this pull request 3 days ago

Fix lora flashinfer import bug on ROCM

github.com/sgl-project/sglang - Fridge003 opened this pull request 3 days ago

Compatibility of SGLang for NVIDIA Xavier

github.com/sgl-project/sglang - rakshit2020 opened this issue 3 days ago

[Bug] Lora Non-Flashinfer backend select error

github.com/sgl-project/sglang - BruceXcluding opened this issue 3 days ago

Update Triton extend backend interface

github.com/sgl-project/sglang - ispobock opened this pull request 3 days ago

Update documents on DeepSeek_v3 server launch args #3283

github.com/sgl-project/sglang - lycanlancelot opened this pull request 3 days ago

[Bug] sglang-router curl get return without `content-type: application/json` in the header

github.com/sgl-project/sglang - bmkor opened this issue 3 days ago

[ROCm] Logic to decide whether to used manually unrolled kernel.

github.com/sgl-project/sglang - whchung opened this pull request 3 days ago

Use forward_cuda to execute custom op for hip platform

github.com/sgl-project/sglang - kkHuang-amd opened this pull request 3 days ago

[Bug] RuntimeError: RMSNorm failed with error code invalid configuration argument

github.com/sgl-project/sglang - YJHMITWEB opened this issue 3 days ago

[Bug] TypeError: AWQMoEMethod.create_weights() missing 1 required positional argument: 'intermediate_size_per_partition' when running deepseek-r1-awq model

github.com/sgl-project/sglang - xihuai18 opened this issue 3 days ago

[Docs Bug] examples/save_sharded_state.py does not exist

github.com/sgl-project/sglang - sshleifer opened this issue 3 days ago

FEA compat with ipv6

github.com/sgl-project/sglang - lxww302 opened this pull request 3 days ago

Fix: Runtime error for function calling

github.com/sgl-project/sglang - shuaills opened this pull request 3 days ago

[ROCm] Manually unroll _w8a8_block_fp8_matmul kernel on AMD GPU.

github.com/sgl-project/sglang - whchung opened this pull request 3 days ago

[Feature] Support llguidance for constrained decoding

github.com/sgl-project/sglang - JC1DA opened this pull request 3 days ago

add date_string to the chat template

github.com/sgl-project/sglang - cctry opened this pull request 3 days ago

[Bug] chat template of Llama 3.1 injects a wrong today's date in the system prompt.

github.com/sgl-project/sglang - cctry opened this issue 3 days ago

[Bug] Structured Output Failed When using Regex and EBNF

github.com/sgl-project/sglang - xihuai18 opened this issue 4 days ago

[ROCm] Add tuning configs for AMD Radeon Graphics.

github.com/sgl-project/sglang - whchung opened this pull request 4 days ago

update flashinfer install index url

github.com/sgl-project/sglang - zhyncs opened this pull request 4 days ago

Update Triton decode backend interface

github.com/sgl-project/sglang - ispobock opened this pull request 4 days ago

[Feature] When running sglang in several nodes, all of them downloads the model from hub simultaneously

github.com/sgl-project/sglang - vabatista opened this issue 4 days ago

[Feature] #3287 follow up

github.com/sgl-project/sglang - HaiShaw opened this issue 4 days ago

[Bug] Fix #3161 break on ROCm

github.com/sgl-project/sglang - HaiShaw opened this issue 4 days ago

upgrade flashinfer v0.2.0.post2

github.com/sgl-project/sglang - zhyncs opened this pull request 4 days ago

ROCm: sgl-kernel enablement starting with sgl_moe_align_block

github.com/sgl-project/sglang - HaiShaw opened this pull request 4 days ago

[Bug] I have observed that vLLM exhibits faster response times compared to SGLang for certain workload

github.com/sgl-project/sglang - bao231 opened this issue 4 days ago

ROCm: sgl-kernel enablement starting with sgl_moe_align_block

github.com/sgl-project/sglang - HaiShaw opened this pull request 4 days ago

[Bug] min_p_sampling_from_probs()

github.com/sgl-project/sglang - QualiaSystemsAI opened this issue 4 days ago

[Bug] deepseek v3 2 nodes h100 segmentation fault

github.com/sgl-project/sglang - victorserbu2709 opened this issue 4 days ago

[Feature] Support compatibility between Cuda Graph and Lora

github.com/sgl-project/sglang - Fridge003 opened this issue 4 days ago

0.4.2.post2 bump

github.com/sgl-project/sglang - HaiShaw opened this pull request 4 days ago

[Bug] DeepSeek R1 cannot run if using 16K input

github.com/sgl-project/sglang - Wesley-Jzy opened this issue 4 days ago

[Bug] I have observed that vLLM exhibits faster response times compared to SGLang for certain workload

github.com/sgl-project/sglang - bao231 opened this issue 4 days ago

[Feature] Support lm eval harness and lighteval to SGLang

github.com/sgl-project/sglang - zhaochenyang20 opened this issue 4 days ago

[Bug] Error when performing function calling: `Exception: Extra data: line 1 column 41`

github.com/sgl-project/sglang - atbe opened this issue 4 days ago

add Atlas Cloud for Adoption and Sponsorship

github.com/sgl-project/sglang - zhyncs opened this pull request 4 days ago

added amd_configure.md to references

github.com/sgl-project/sglang - zstreet87 opened this pull request 4 days ago

add Nebius for Adoption and Sponsorship

github.com/sgl-project/sglang - zhyncs opened this pull request 4 days ago

Update server args doc

github.com/sgl-project/sglang - simveit opened this pull request 4 days ago