github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

Fix the race condition in overlap mode

merrymercy opened this pull request 3 months ago

Update vllm to 0.6.3

ispobock opened this pull request 3 months ago

Fix `is_all_ready` for overlap copy

merrymercy opened this pull request 3 months ago

Simplify the nan detection and greedy check in sampler

merrymercy opened this pull request 3 months ago

Does frontend language support multi-image QA?

joeyy5588 opened this issue 3 months ago

Skip unnecessary penalizer

merrymercy opened this pull request 3 months ago

Add grouped free operations

merrymercy opened this pull request 3 months ago

Add dtype for more operations

merrymercy opened this pull request 3 months ago

Simplify flashinfer utilities

merrymercy opened this pull request 3 months ago

Fix regex and logprob conflicts when chunked prefilling

hnyls2002 opened this pull request 3 months ago

Fix mixed batch for multi modal models

merrymercy opened this pull request 3 months ago

Fix engine unit test

merrymercy opened this pull request 3 months ago

Fix failed ci tests on long prompts; Better error messages for embedding models

merrymercy opened this pull request 3 months ago

Fix the failed unit tests

merrymercy opened this pull request 3 months ago

[Bug] AttributeError in `openai.Client` Embeddings API

tanzelin430 opened this issue 3 months ago

feat: radix tree code optimize

wxsms opened this pull request 3 months ago

Use SGLang imports for linear layer

janimo opened this pull request 3 months ago

[Router] Implement router backbone

ByronHsu opened this pull request 3 months ago

ORJson. Faster Json serialization

michaelfeil opened this pull request 3 months ago

[Bug] crash about `c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout`

zeng-zc opened this issue 3 months ago

[Bug] IndexError: Inconsistent batch_size and len(image_input)

OBJECT907 opened this issue 3 months ago

[Bug] deadlock or hang on Qwen2-7B models

zeng-zc opened this issue 3 months ago

Update the transformers version in CI

merrymercy opened this pull request 3 months ago

Update README.md

merrymercy opened this pull request 3 months ago

add orjson for jsonresponse

michaelfeil opened this pull request 3 months ago

Launch a thread to overlap CPU and GPU

merrymercy opened this pull request 3 months ago

[Event] Add online meetup meeting link

Ying1123 opened this pull request 3 months ago

Fix srt dependency

ispobock opened this pull request 3 months ago

Add matched_stop token or str to distinguish between eos or stop str finish_reason generation

g-drozdov opened this pull request 3 months ago

[Bug] ROCm6.1.2 sglang0.3.3 cuda graph coredump

linqingxu opened this issue 3 months ago

Fixes for running reward model inference using sglang

corbt opened this pull request 3 months ago

Fix filter_batch function call

hnyls2002 opened this pull request 3 months ago

[Performance] Support `xgrammar` for faster constrained decoding

DarkSharpness opened this pull request 3 months ago

Add date to logging messages (#1623)

zeng-zc opened this pull request 3 months ago

slides link to .pdf

ziliangpeng opened this pull request 3 months ago

Add a new event loop

merrymercy opened this pull request 3 months ago

Add OLMo model

janimo opened this pull request 3 months ago

Fix memory leak during abort

merrymercy opened this pull request 3 months ago

[Feature] Make vLLM optional in model code

ByronHsu opened this issue 3 months ago

Improve benchmark scripts

merrymercy opened this pull request 3 months ago

[Minor] Add some utility functions

merrymercy opened this pull request 3 months ago

[doc] improve engine doc and add to readme

ByronHsu opened this pull request 3 months ago

[Feature] When will a version of S-Lora be available?

kunkunzhang123 opened this issue 3 months ago

Simplify chunked prefill

merrymercy opened this pull request 3 months ago

[Minor] Improve style

merrymercy opened this pull request 3 months ago

Fix unit test order to balance the tasks in CI

merrymercy opened this pull request 3 months ago

[Bug] difference of kv-cache-prefixing between vLLM and sglang

chenchunhui97 opened this issue 3 months ago

Move filter_batch out of stream_output

merrymercy opened this pull request 3 months ago

Add a test case to test retract

merrymercy opened this pull request 3 months ago

[Minor] Rename no_eos_trim to no_stop_trim

Ying1123 opened this pull request 3 months ago

docs: add zh_CN po files

llama-factory opened this pull request 3 months ago

Add output_ids into ScheduleBatch

merrymercy opened this pull request 3 months ago

[1/N] Remove `CacheConfig` import in all model files

ByronHsu opened this pull request 3 months ago

temp

yukavio opened this pull request 3 months ago

[doc] Add engine section in backend.md

ByronHsu opened this pull request 3 months ago

[Feature] sanic Custom Server example support openai stream api ?

lys791227 opened this issue 3 months ago

Fix the batch_is_full check for jump-forward decoding

merrymercy opened this pull request 3 months ago

Add get_tokenizer function for Engine class

pjyi2147 opened this pull request 3 months ago

Simplify the event loop and expose `--num-continuous-decode-steps` as an argument

merrymercy opened this pull request 3 months ago

Add an option to disable penalizer

merrymercy opened this pull request 3 months ago

[Fix] fix eos trim inconsistency

Ying1123 opened this pull request 3 months ago

[Feature] Multi-instance deployment

vkc1vk opened this issue 3 months ago

Fix unit tests and type annotations

merrymercy opened this pull request 3 months ago

docs: add zh_CN po files

llama-factory opened this pull request 3 months ago

dead

llama-factory opened this pull request 3 months ago

Fix ignore_eos in the OpenAI ChatCompletions API

merrymercy opened this pull request 3 months ago

Surpport kv cache int8/int4 for triton backend

yuguo-Jack opened this pull request 3 months ago

[Feature] Support for GPT-2

hariharan98m opened this issue 3 months ago

fix missing ignore_eos in v1/chat/completions

learninmou opened this pull request 3 months ago

Add GLM-4v Multimodal Model support for SGLang

sixsixcoder opened this pull request 3 months ago

is it time to rerun the benchmarks?

stas00 opened this issue 3 months ago

[Fix] Fix the style of test_large_max_new_tokens.py

merrymercy opened this pull request 3 months ago

Increase backend-test2 timeout from 20 to 25 mins

ByronHsu opened this pull request 3 months ago

Release v0.3.3.post1

merrymercy opened this pull request 3 months ago

Add back data parallelism

merrymercy opened this pull request 3 months ago

Fix constrained decoding

merrymercy opened this pull request 3 months ago

Added a "Back To Top" Button

JanumalaAkhilendra opened this pull request 3 months ago

Add image_token in conversation.py

merrymercy opened this pull request 3 months ago

Fix the correctness test in bench_latency.py when tp > 1 and test_generation_models.py

merrymercy opened this pull request 3 months ago

[Bug] The time unit in bench_serving is wrong on A800-SXM4-40GB, using perf_counter_ns could fix

zeng-zc opened this issue 3 months ago

Update README.md

merrymercy opened this pull request 3 months ago

Add the ability to enable and disable the Profiler via HTTP API.

abatom opened this pull request 3 months ago

Update README.md

kushal34712 opened this pull request 3 months ago

[Feature] Using frontend APIs but passing a list of prompts in `run` rather than `run_batch`

pengye91 opened this issue 3 months ago

[Feature] Add date to logging

zeng-zc opened this issue 3 months ago

[Bug] JSON Regex does not work for vision model

ByronHsu opened this issue 3 months ago

Fix test_vision_openai_server on CI

ByronHsu opened this pull request 3 months ago

[Performance, hardware] MoE tuning update to AMD MI300x GPUs

HaiShaw opened this pull request 3 months ago

[Question]Where is data parallelism

josephydu opened this issue 3 months ago

[Bug] Message to guide using <=0.3.2 for data parallel is not shown when --dp is set

seungduk-yanolja opened this issue 3 months ago

[Feature] GGUF support

remixer-dec opened this issue 3 months ago

[Minor] Fix logging typo

amosyou opened this pull request 3 months ago

[engine] support async and streaming

ByronHsu opened this pull request 3 months ago

[Bug] SGLang hangs after hitting 0.00 token usage on Engine.generate

pjyi2147 opened this issue 3 months ago

Nit about the decorator of `PortArgs.init_new`

glen-amd opened this pull request 3 months ago

[Bug] Can't access one click llms on runpod

kovern opened this issue 3 months ago

[Bug] Can't distinguish whether the EOS token or the stop token caused the generation to stop

g-drozdov opened this issue 3 months ago

[Feature] Log input text instead of input_ids when using openai chat apis

CedricHwong opened this issue 3 months ago

Add device support

liangan1 opened this pull request 3 months ago

[Engine] Fix generate hanging issue after the first call

ByronHsu opened this pull request 3 months ago