github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

[Feature] Support Qwen2-VL based embedding model

VoVAllen opened this issue about 2 months ago

Github runner instructions for AMD

HaiShaw opened this pull request about 2 months ago

benchmark json schema

DarkSharpness opened this pull request about 2 months ago

[Bug] Does Mixtral currently not support torch compile?

sitabulaixizawaluduo opened this issue about 2 months ago

chore: open lto and optimization in release profile

ethe opened this pull request about 2 months ago

Add download_dir ServerArgs property

pjyi2147 opened this pull request about 2 months ago

set content to empty string

chottolabs opened this pull request about 2 months ago

[BUG] Jump forward w/ outlines backend slightly changes the decoding results

merrymercy opened this issue about 2 months ago

Fix dependency and error message for xgrammar

merrymercy opened this pull request about 2 months ago

Do not let invalid grammar crash the server

merrymercy opened this pull request about 2 months ago

Release v0.3.5.post1

merrymercy opened this pull request about 2 months ago

[Bug] Have any suggestions for setting hyperparameters for inference acceleration？

948024326 opened this issue about 2 months ago

Fix grammar backend for tensor parallelism

merrymercy opened this pull request about 2 months ago

[WIP] [Router] Multi Tenant Radix Tree

ByronHsu opened this pull request about 2 months ago

Refactor grammar backend

merrymercy opened this pull request about 2 months ago

[BUG] xgrammar does not follow the constraint

merrymercy opened this issue about 2 months ago

[WIP] Use FlashInfer RoPE

james-p-xu opened this pull request about 2 months ago

fix test_embedding_models prompt length too long's bug

BBuf opened this pull request about 2 months ago

fix a bug in v1_embeeding_request

BBuf opened this pull request about 2 months ago

Fix finish reason

merrymercy opened this pull request about 2 months ago

poke test

ByronHsu opened this pull request about 2 months ago

Filter empty prompt in random bench serving

ispobock opened this pull request about 2 months ago

cleanup rust folder

ByronHsu opened this pull request about 2 months ago

Fix weight loading for tied word embedding when TP > 1

merrymercy opened this pull request about 2 months ago

Fix a typo in io_struct.py

merrymercy opened this pull request about 2 months ago

[Feature] Regex stop condition

SinanAkkoyun opened this issue about 2 months ago

[Minor] Remove unused imports

merrymercy opened this pull request about 2 months ago

fix sglang_router not found

ByronHsu opened this pull request about 2 months ago

Bump router to 0.0.3

ByronHsu opened this pull request about 2 months ago

run rust test on ubuntu instead of 1-gpu-runner

ByronHsu opened this pull request about 2 months ago

release router from py38 to py312

ByronHsu opened this pull request about 2 months ago

Fix rust unit test and pypi token

ByronHsu opened this pull request about 2 months ago

Add Engine::encode example

james-p-xu opened this pull request about 2 months ago

support echo=true and logprobs in openai api when logprobs=1 in lm-evaluation-harness

BBuf opened this pull request 2 months ago

support parallel grammar preprocessing

DarkSharpness opened this pull request 2 months ago

support internlm2-reward

RangiLyu opened this pull request 2 months ago

[Feature] Does sglang support only input embeds?

OswaldoBornemann opened this issue 2 months ago

[Feature] Are there plans to support AWQ and torch compile?

sitabulaixizawaluduo opened this issue 2 months ago

[Feature] Enable special token parsing of OAI messages

SinanAkkoyun opened this issue 2 months ago

[CI] Balance unit tests

merrymercy opened this pull request 2 months ago

[Bug] how to combine with ray.data

mdy666 opened this issue 2 months ago

docs: add shm size for docker run

zhyncs opened this pull request 2 months ago

[Bug] Run llava 1.5 backend get an error

pspdada opened this issue 2 months ago

qwen2vl fix bug for #1971 #1897

yizhang2077 opened this pull request 2 months ago

fix: update pyzmq version

zhyncs opened this pull request 2 months ago

Specify `zmq` Version Requirement

HuanzhiMao opened this pull request 2 months ago

Simplify prometheus metrics

merrymercy opened this pull request 2 months ago

[Performance, Triton] Optimize over mask compute to tl.load in fused_moe_kernel

HaiShaw opened this pull request 2 months ago

[Feature] Throughput-aware speculative decoding

vkc1vk opened this issue 2 months ago

[Feature] Is GritLM-7B supported?

vkc1vk opened this issue 2 months ago

[CI] balance unit tests

merrymercy opened this pull request 2 months ago

[Minor] Fix a typo in test_torchao.py

merrymercy opened this pull request 2 months ago

Update pr-test-rust.yml to add a "finish" step

merrymercy opened this pull request 2 months ago

Update README.md

merrymercy opened this pull request 2 months ago

Initialize model_worker_batch variable

qeternity opened this pull request 2 months ago

Clean up metrics code

merrymercy opened this pull request 2 months ago

multimodal can not use the choices?

luowei0701 opened this issue 2 months ago

Support DP MLA

ispobock opened this pull request 2 months ago

test

Sanger2000 opened this pull request 2 months ago

Offline LLM Engine Benchmark Throughput

zolinthecow opened this pull request 2 months ago

Updated Instructions on Profiling SGLang Infer System with AMD GPUs

leishaoSC opened this pull request 2 months ago

[Feature] Is AWQ W4Afp8 supported?

vkc1vk opened this issue 2 months ago

Fix metrics

binarycrayon opened this pull request 2 months ago

Update README.md's Slack invitation link

zhaochenyang20 opened this pull request 2 months ago

[minor] Improve code style and compatibility

merrymercy opened this pull request 2 months ago

[Bug] Inference with RadixAttention，but output weirdly

walker-ai opened this issue 2 months ago

Add sentence_transformers to CI dependency

merrymercy opened this pull request 2 months ago

[Release, ROCm] release ROCm docker build for AMD MI GPUs

HaiShaw opened this pull request 2 months ago

Adjust reward model's score module and pooler module order for reducing computation

aqweteddy opened this pull request 2 months ago

Remove the useless to_srt_kwargs

merrymercy opened this pull request 2 months ago

Gemma2 reward model support

aqweteddy opened this pull request 2 months ago

[Bug] amdgpu，tp-size=2，Detected errors during sampling! NaN in the logits.

linqingxu opened this issue 2 months ago

Update setup_github_runner.md

merrymercy opened this pull request 2 months ago

Add a timeout for execute-notebook.yml

merrymercy opened this pull request 2 months ago

how to load a gguf model in sglang?

lovingliferwj opened this issue 2 months ago

[Doc] fix docs

merrymercy opened this pull request 2 months ago

Add model support for Phi 3.5 MoE

svaruag opened this pull request 2 months ago

Concurrency Issue: Multiple Requests Not Being Processed Simultaneously

hahmad2008 opened this issue 2 months ago

[Bug] tp-size=2，model launch error

linqingxu opened this issue 2 months ago

[Bug] http_request Function Causing 403 Error

tanushmahalka opened this issue 2 months ago

[Bug] Issue with reward model API

dmakhervaks opened this issue 2 months ago

[Docs] fix 404 - Contributor Guide

HaiShaw opened this pull request 2 months ago

[Performance, Triton Kernel Args] extend_attention, optimize kern args to _fwd_kernel

HaiShaw opened this pull request 2 months ago

fix black in pre-commit

zhaochenyang20 opened this pull request 2 months ago

[ENV, ROCm] update environment settings

HaiShaw opened this pull request 2 months ago

ci: enable `black-jupyter` in pre-commit CI

XuehaiPan opened this pull request 2 months ago

[Feature] How to serve GGUF model?

hahmad2008 opened this issue 2 months ago

[Feature] Add LoRA Support for Chat Completion in SGLang

mssongit opened this issue 2 months ago

[rust] cache-aware DP - approx tree

ByronHsu opened this pull request 2 months ago

Monitoring documentation

binarycrayon opened this pull request 2 months ago

[Feature] Save cache from requests and load

SinanAkkoyun opened this issue 2 months ago

[Bug] Seeing random output with nvidia/Llama-3.1-Nemotron-70B-Reward

pgimenes opened this issue 2 months ago

[Bug] Incompatible with outlines>=0.1.0

dzimmerman-nci opened this issue 2 months ago

Instructions on Profiling SGLang Infer System with AMD GPUs

leishaoSC opened this pull request 2 months ago

fix url in ipv6-only when warm-up

cauyxy opened this pull request 2 months ago

[Bug] Get connection error when use sglang python module

kanebay opened this issue 2 months ago

minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces

XuehaiPan opened this pull request 2 months ago

[Bug] Torch 2.5 issue with Tensor Parallel Size > 1

CortexEdgeUser opened this issue 2 months ago

[Doc] improve relative links and structure

merrymercy opened this pull request 2 months ago

[Bug] Launching a server with `--enable-torch-compile` produce torch dynamo error

msublee opened this issue 2 months ago