SGLang issues | Ecosyste.ms: OpenCollective

[CI] Print summary on github actions

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Kernel] Launch two kernels for mixed chunked prefill

github.com/sgl-project/sglang - merrymercy opened this issue about 1 month ago

[Kernel] cuDNN attention backend

github.com/sgl-project/sglang - merrymercy opened this issue about 1 month ago

[Kernel] Optimize triton decoding kernels for long context

github.com/sgl-project/sglang - merrymercy opened this issue about 1 month ago

[Feature] support gptq or awq for deepseek v2

github.com/sgl-project/sglang - Xu-Chen opened this issue about 1 month ago

Add new contributors so they can trigger CI automatically

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Fix the default chunked prefill size

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

update weights from distributed

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

add get weights by parameter name for llama

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

udate weights from disk

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

Sgl online weights update [WIP]

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

[Bug] Image Prompts crashing sglang server in environment without Internet (llama 3.2)

github.com/sgl-project/sglang - adamocarolli opened this issue about 1 month ago

Get parameter by name

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

minor: add sgl-kernel dir

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

rename update weights from disk api

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

chore: bump v0.3.6.post3

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[Bug] flashinfer's RMSNorm implementation causes precision differences in model outputs compared to the HuggingFace implementation

github.com/sgl-project/sglang - BBuf opened this issue about 1 month ago

[Minor] fix the style for multimodal models

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Fix hash collision for multi modal models

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Bug] AssertionError: assert self.being_chunked_req is None, related to `chunked_prefill_size` and input_len

github.com/sgl-project/sglang - HaiShaw opened this issue about 1 month ago

Simplify tokenizer manager

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Revert "Revert "Add simple CPU offloading support""

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

Revert "Add simple CPU offloading support"

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

Update backend.md

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Update backend.md

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Feature] Windows OS support sglang

github.com/sgl-project/sglang - Zhangwt95 opened this issue about 1 month ago

Update ModelRunner Weights From Distributed

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

Openai api supports lora path

github.com/sgl-project/sglang - ccchow opened this pull request about 1 month ago

[question] i wander what's the normal speed of sglang at a 8 * a800 machine.

github.com/sgl-project/sglang - guox18 opened this issue about 1 month ago

[Track] progress in removing vLLM dependencies

github.com/sgl-project/sglang - zhyncs opened this issue about 1 month ago

adapt vllm distributed module to sglang

github.com/sgl-project/sglang - yizhang2077 opened this pull request about 1 month ago

Support LoRA in Completion API

github.com/sgl-project/sglang - bjmsong opened this pull request about 1 month ago

fix missing launch server import

github.com/sgl-project/sglang - qeternity opened this pull request about 1 month ago

Add a simple torch native attention backend

github.com/sgl-project/sglang - YangQun1 opened this pull request about 1 month ago

Why does the number of requests per second of 8 a800 machines not increase linearly compared to 1 a800 machine?

github.com/sgl-project/sglang - tensorflowt opened this issue about 1 month ago

[fix] Fix prefix caching for multi-image/video

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

Fix memory leak during abort

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Feature] QwQ support

github.com/sgl-project/sglang - zhyncs opened this issue about 1 month ago

[fix] Fix multi-image/video with prefix caching

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

Disable overlap scheduler for multimodal models

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Fix chunked prefill size for bench_offline_throughput

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Add OLMo2 model.

github.com/sgl-project/sglang - janimo opened this pull request about 1 month ago

Update Install Method 2. From source

github.com/sgl-project/sglang - HaiShaw opened this pull request about 1 month ago

Crash the server correctly during error

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Lint

github.com/sgl-project/sglang - BBuf opened this pull request about 1 month ago

Update weights from dist [WIP]

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

Rename tuned MI300X config files for fused_moe_triton

github.com/sgl-project/sglang - HaiShaw opened this pull request about 1 month ago

Bump sglang-router to 0.0.10 for env name change

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[benchmark] Add fused_moe_triton benchmark and tuning tools

github.com/sgl-project/sglang - BBuf opened this pull request about 1 month ago

fix typo prompts

github.com/sgl-project/sglang - qibaoyuan opened this pull request about 1 month ago

Remove fused_moe_grok

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Revert "Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default"

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Update model_loader deps and qqq quantization deps

github.com/sgl-project/sglang - HandH1998 opened this pull request about 1 month ago

[Feature] support torchao for qwen2 models

github.com/sgl-project/sglang - tricky61 opened this issue about 1 month ago

Rename DP_RANK to SGLANG_DP_RANK

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Bug] (v0.3.6.post2) Output degredation when using structured output

github.com/sgl-project/sglang - Quang-elec44 opened this issue about 1 month ago

[FEAT] Support GGUF format

github.com/sgl-project/sglang - zhengy001 opened this pull request about 1 month ago

Release v0.3.6.post2

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Feature] ROCm in WSL environment is supported

github.com/sgl-project/sglang - githust66 opened this issue about 1 month ago

Fix flasky tests

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

为啥这里是异步拷贝，然后后面直接使用了？没有显示同步？

github.com/sgl-project/sglang - MichoChan opened this issue about 1 month ago

[feat] Support session control for vision language models

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

[Bug] (Xgrammar) (v0.3.6.post1) RuntimeError: CUDA dependencies are not installed.

github.com/sgl-project/sglang - Quang-elec44 opened this issue about 1 month ago

Fix rust warning

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Bump router to 0.0.9 with better logging

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

add sglang version to get_server_info

github.com/sgl-project/sglang - binarycrayon opened this pull request about 1 month ago

[Feature] Add sglang version to server info

github.com/sgl-project/sglang - binarycrayon opened this issue about 1 month ago

docs: update adoption

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

MoE Expert Parallel Impl

github.com/sgl-project/sglang - xiaobochen123 opened this pull request about 1 month ago

minor: update check_env

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[Feature] 2:4 sparsity support

github.com/sgl-project/sglang - arunpatala opened this issue about 1 month ago

fix: add cuda-python for xgrammar

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Remove unresolved reference 'self'

github.com/sgl-project/sglang - apemost opened this pull request about 1 month ago

Update sampler.py to skip the success check

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

using `is not` not `!=` to test `None`

github.com/sgl-project/sglang - WrRan opened this pull request about 1 month ago

remove unused imports

github.com/sgl-project/sglang - WrRan opened this pull request about 1 month ago

[Question] RuntimeError: Initialization failed

github.com/sgl-project/sglang - LiYuhang9527 opened this issue about 1 month ago

Support top n sigma sampling

github.com/sgl-project/sglang - Snowdar opened this pull request about 1 month ago

[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs

github.com/sgl-project/sglang - kkHuang-amd opened this pull request about 1 month ago

[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs.

github.com/sgl-project/sglang - kkHuang-amd opened this pull request about 1 month ago

Release v0.3.6.post1

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Rename double sparsity config file

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[CI] Minor fix for CI

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Update CI threshold

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Use custom allreduce w/ torch.compile

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[router] Rust e2e test

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[router] Replace print with logger

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Bump rustls from 0.23.16 to 0.23.18 in /rust

github.com/sgl-project/sglang - dependabot[bot] opened this pull request about 1 month ago

[Bug] Qwen2-VL-7B IndexError

github.com/sgl-project/sglang - jakep-allenai opened this issue about 1 month ago

[CI] Split test cases in CI for better load balancing

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

feat: add should_use_tensor_core

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[Feature] Get the real logprobs to analyze decoding

github.com/sgl-project/sglang - Snowdar opened this issue about 1 month ago

[Bug] frequency penalty

github.com/sgl-project/sglang - vivian0429 opened this issue about 1 month ago

Update XGrammar to the latest API

github.com/sgl-project/sglang - Ubospica opened this pull request about 1 month ago

[Fix] Avoid calling fill_vocab_mask for terminated requests

github.com/sgl-project/sglang - Ubospica opened this pull request about 1 month ago

feat: fused_moe fp8 monkey patch

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[feat] Refactor session control interface and add CI

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

Question about ragged wrapper

github.com/sgl-project/sglang - ZhongYingMatrix opened this issue about 1 month ago