SGLang issues | Ecosyste.ms: OpenCollective

minor: update check_env

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[Feature] 2:4 sparsity support

github.com/sgl-project/sglang - arunpatala opened this issue about 1 month ago

fix: add cuda-python for xgrammar

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Remove unresolved reference 'self'

github.com/sgl-project/sglang - apemost opened this pull request about 1 month ago

Update sampler.py to skip the success check

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

using `is not` not `!=` to test `None`

github.com/sgl-project/sglang - WrRan opened this pull request about 1 month ago

remove unused imports

github.com/sgl-project/sglang - WrRan opened this pull request about 1 month ago

[Question] RuntimeError: Initialization failed

github.com/sgl-project/sglang - LiYuhang9527 opened this issue about 1 month ago

Support top n sigma sampling

github.com/sgl-project/sglang - Snowdar opened this pull request about 1 month ago

[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs

github.com/sgl-project/sglang - kkHuang-amd opened this pull request about 1 month ago

[3rdparty, document] Updated Documentation that for triton fused_moe kernel tuning for AMD Instinct GPUs.

github.com/sgl-project/sglang - kkHuang-amd opened this pull request about 1 month ago

Release v0.3.6.post1

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Rename double sparsity config file

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[CI] Minor fix for CI

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Update CI threshold

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Use custom allreduce w/ torch.compile

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[router] Rust e2e test

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[router] Replace print with logger

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Bump rustls from 0.23.16 to 0.23.18 in /rust

github.com/sgl-project/sglang - dependabot[bot] opened this pull request about 1 month ago

[Bug] Qwen2-VL-7B IndexError

github.com/sgl-project/sglang - jakep-allenai opened this issue about 1 month ago

[CI] Split test cases in CI for better load balancing

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

feat: add should_use_tensor_core

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[Feature] Get the real logprobs to analyze decoding

github.com/sgl-project/sglang - Snowdar opened this issue about 1 month ago

[Bug] frequency penalty

github.com/sgl-project/sglang - vivian0429 opened this issue about 1 month ago

Update XGrammar to the latest API

github.com/sgl-project/sglang - Ubospica opened this pull request about 1 month ago

[Fix] Avoid calling fill_vocab_mask for terminated requests

github.com/sgl-project/sglang - Ubospica opened this pull request about 1 month ago

feat: fused_moe fp8 monkey patch

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[feat] Refactor session control interface and add CI

github.com/sgl-project/sglang - Ying1123 opened this pull request about 1 month ago

Question about ragged wrapper

github.com/sgl-project/sglang - ZhongYingMatrix opened this issue about 1 month ago

[Performance]: Process affinity to CPU cores with multiple sockets support

github.com/sgl-project/sglang - HaiShaw opened this pull request about 1 month ago

Replace prob based with threshold based load balancing

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Allow overwrite flashinfer use_tensorcore

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Feature] How to accelerate constrained decoding when regex needs to change with input?

github.com/sgl-project/sglang - GrittyChen opened this issue about 1 month ago

[Fused moe] add tuning fused configs for qwen2 57b and mixtral 8x7b

github.com/sgl-project/sglang - BBuf opened this pull request about 1 month ago

[Bug] cannot import name 'CachedGrammarCompiler' from 'xgrammar' (version 0.3.6)

github.com/sgl-project/sglang - Quang-elec44 opened this issue about 1 month ago

Test indeterminism of sgl.select under high concurrency

github.com/sgl-project/sglang - qeternity opened this pull request about 1 month ago

Fix docs

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Rename triton_fused_moe -> fused_moe_triton

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Balance CI tests

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

fix: use torch.sum for compatible

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

[Bug] FusedMoE compatible with vllm 0.6.3.post1

github.com/sgl-project/sglang - zhyncs opened this issue about 1 month ago

Update CI threshold & Improve code style

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Fix mixed chunked prefill in overlap mode

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

fix: resolve end-of-file-fixer

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

feat: update other MoE models deps

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

feat: update gitignore and add tuning config for FusedMoE

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Simplify `Scheduler.update_running_batch`

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

feat: remove the dependency on FusedMoE

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Merged three native APIs into one: get_server_info

github.com/sgl-project/sglang - henryhmko opened this pull request about 1 month ago

[Bug] llava use image hash as token，leading to cache bug

github.com/sgl-project/sglang - zwc163 opened this issue about 1 month ago

Eagle speculative decoding part 4: Add EAGLE2 worker

github.com/sgl-project/sglang - yukavio opened this pull request about 1 month ago

Byhsu/fairness router

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Improve sglang router

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

add prefix match for certain tenant

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Add more api routes (completion, health, etc) to the router

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[Draft] Resolving integration differences after XGrammar lauch refactoring

github.com/sgl-project/sglang - gittb opened this pull request about 1 month ago

fix dp_rank env

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

update router doc

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

Bump sglang-router to 0.0.5

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[Bug] Error when using LLAVA 1.5 for llava bench

github.com/sgl-project/sglang - pspdada opened this issue about 1 month ago

fix: resolve bench_serving args

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Fix dp print message

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[CI] Fix test cases

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Add concurrency option for benchmark

github.com/sgl-project/sglang - cermeng opened this pull request about 1 month ago

Add concurrency option in benchmark

github.com/sgl-project/sglang - cermeng opened this pull request about 1 month ago

Fix grid size in Triton decoding kernel

github.com/sgl-project/sglang - ispobock opened this pull request about 1 month ago

[Bug] Error when launching llava1.5

github.com/sgl-project/sglang - pspdada opened this issue about 1 month ago

deps(flashinfer): fix `is_flashinfer_available()` and make `flashinfer` optional dependency

github.com/sgl-project/sglang - XuehaiPan opened this pull request about 1 month ago

[Feature] Support LLaMA-3.2 finetuned with Sentence Transformers !

github.com/sgl-project/sglang - thusinh1969 opened this issue about 1 month ago

Revert "Only stream output on tp rank 0"

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

EAGLE2: general part [2]

github.com/sgl-project/sglang - yukavio opened this pull request about 1 month ago

EAGLE2: Eagle related part [1]

github.com/sgl-project/sglang - yukavio opened this pull request about 1 month ago

feat(pre-commit): trim unnecessary notebook metadata from git history

github.com/sgl-project/sglang - XuehaiPan opened this pull request about 1 month ago

fix: add xgrammar dependency

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

minor: update gsm8k threshold

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Only stream output on tp rank 0

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

add profile in offline benchmark & update doc

github.com/sgl-project/sglang - bjmsong opened this pull request about 1 month ago

[minor] Clean up unused imports

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

Add initial support for intel Gaudi accelerators

github.com/sgl-project/sglang - ankurneog opened this pull request about 1 month ago

chore: bump v0.3.6

github.com/sgl-project/sglang - zhyncs opened this pull request about 1 month ago

Online weight update [WIP]

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request about 1 month ago

Rename sglang.bench_latency to sglang.bench_one_batch

github.com/sgl-project/sglang - merrymercy opened this pull request about 1 month ago

[Bug] Unable to load GPTQ Mixtral 8x7 v0.1 with SGLang

github.com/sgl-project/sglang - DhruvaBansal00 opened this issue about 1 month ago

Turn off autotune for scaled mm for fp8 dynamic quant in torchao

github.com/sgl-project/sglang - jerryzh168 opened this pull request about 1 month ago

[router] add base_gpu_id server args & merged radix tree python reference

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[router] cache-aware load-balancing router v1

github.com/sgl-project/sglang - ByronHsu opened this pull request about 1 month ago

[Feature] Inference example code for Qwen2-VL

github.com/sgl-project/sglang - YuanLiuuuuuu opened this issue about 1 month ago

[Bug] Qwen2-VL-7B with sglang Performance Degradation on MME benchmark

github.com/sgl-project/sglang - Mr-Loevan opened this issue about 2 months ago

ROCm: Fix MoE padding for none FP8 cases

github.com/sgl-project/sglang - HaiShaw opened this pull request about 2 months ago

Benchmark with Pytorch Profiler easily

github.com/sgl-project/sglang - bjmsong opened this pull request about 2 months ago

[Feature] Support for rerank models

github.com/sgl-project/sglang - dinhanhx opened this issue about 2 months ago

[Feature] Is Yarn supported in sglang?

github.com/sgl-project/sglang - klykq111 opened this issue about 2 months ago

Error out when torchao-config option is not recognized

github.com/sgl-project/sglang - jerryzh168 opened this pull request about 2 months ago

Fix #2037 - Context length check does not take into out pad tokens for visual models

github.com/sgl-project/sglang - jakep-allenai opened this pull request about 2 months ago

Enable overlap scheduler by default for the triton attention backend

github.com/sgl-project/sglang - merrymercy opened this pull request about 2 months ago

Move test_session_id.py to playground

github.com/sgl-project/sglang - merrymercy opened this pull request about 2 months ago

Allow skipping warmup in bench_offline_throughput.py

github.com/sgl-project/sglang - merrymercy opened this pull request about 2 months ago

[Bug] RuntimeError: Failed to allocate memory for batch_prefill_tmp_v with size 435814400 and alignment 16 in AlignedAllocator

github.com/sgl-project/sglang - yuki252111 opened this issue about 2 months ago

feat: use cascade attention kernel (single level)

github.com/sgl-project/sglang - james-p-xu opened this pull request about 2 months ago

Update nightly-eval.yml

github.com/sgl-project/sglang - merrymercy opened this pull request about 2 months ago