Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

SGLang

SGLang is a fast serving framework for large language models and vision language models.
Collective - Host: opensource - https://opencollective.com/sglang - Code: https://github.com/sgl-project/sglang

Qwen2vl support cuda graph and disable radix cache

github.com/sgl-project/sglang - yizhang2077 opened this pull request 2 months ago
[Fix] Fix NaN issues by fixing the cuda graph padding values for flashinfer

github.com/sgl-project/sglang - merrymercy opened this pull request 2 months ago
check user-specified model_max_len with hf derived max_model_len

github.com/sgl-project/sglang - BBuf opened this pull request 2 months ago
[Bug] Catch any errors caused by parsing json schema

github.com/sgl-project/sglang - zolinthecow opened this pull request 3 months ago
Fix MockTokenizer in the unit tests

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix the perf regression due to additional_stop_token_ids

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Crash the server on warnings in CI

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix out of memory message.

github.com/sgl-project/sglang - hnyls2002 opened this pull request 3 months ago
Fix missing additional_stop_token_ids

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Update docs

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Fix] Fix abort in data parallelism

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix stop condition for <|eom_id|>

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix perf regression for set_kv_buffer

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Detected errors during sampling! NaN in the probability error in Qwen2.5-7b-instruct with two a30

github.com/sgl-project/sglang - zzh-www opened this issue 3 months ago
[Feature] Request to 8-bit Quantization of Attention with SageAttention

github.com/sgl-project/sglang - Snowdar opened this issue 3 months ago
[Feature] Multi options

github.com/sgl-project/sglang - QinghanLai opened this issue 3 months ago
[API] add get memory pool size

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
[Bug] Unable to run Qwen2-VL with OpenAI server

github.com/sgl-project/sglang - Quang-elec44 opened this issue 3 months ago
Fuse more ops & Simplify token mapping

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add send request ipynb

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 3 months ago
Add Send request.ipynb

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 3 months ago
Why StreamingResponse 3s Delay to Abort Requests?

github.com/sgl-project/sglang - matthew-hippocratic opened this issue 3 months ago
minor: add human eval

github.com/sgl-project/sglang - zhyncs opened this pull request 3 months ago
[Performance] Support both xgrammar and outlines for constrained decoding

github.com/sgl-project/sglang - DarkSharpness opened this pull request 3 months ago
[Bug] Cannot run `microsoft/Phi-3.5-mini-instruct`; Capture cuda graph failed

github.com/sgl-project/sglang - HuanzhiMao opened this issue 3 months ago
[Bug] Llama 3.1/3.2 model in FC mode output continue past where it should stop

github.com/sgl-project/sglang - HuanzhiMao opened this issue 3 months ago
Release v0.3.4.post1

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Update `max_req_len` and `max_req_input_len`

github.com/sgl-project/sglang - hnyls2002 opened this pull request 3 months ago
Fix edge case for truncated

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
Fix sliding window attention and gemma-2 unit tests in CI

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Introducing SGLang Guru on Gurubase.io

github.com/sgl-project/sglang - kursataktas opened this pull request 3 months ago
[Bug] Issue in latest sglang docker image

github.com/sgl-project/sglang - shubhamgajbhiye1994 opened this issue 3 months ago
Fix prefill oom

github.com/sgl-project/sglang - hnyls2002 opened this pull request 3 months ago
[Bug] when sglang received over 16 concurrency request(means i create 16 thread to call the service all the time), it will return abnormal result, and in the log, will occur NaN

github.com/sgl-project/sglang - GGBond8488 opened this issue 3 months ago
Maintain seq_lens_sum to make more FlashInfer operations non-blocking

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Make token mapping non-blocking in the overlapped mode

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Bug] Prefill OOM!

github.com/sgl-project/sglang - yichuan520030910320 opened this issue 3 months ago
Faster overlap mode scheduler

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
misc: add CODEOWNERS

github.com/sgl-project/sglang - zhyncs opened this pull request 3 months ago
Add GLM-4 TextGeneration Model support for SGLang

github.com/sgl-project/sglang - sixsixcoder opened this pull request 3 months ago
Simplify batch result resolution

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Simplify the usage of device

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add documentations for Installation

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 3 months ago
[Feature] Cache-aware Data Parallel Router

github.com/sgl-project/sglang - ByronHsu opened this issue 3 months ago
Optimize ZMQ receive operations to reduce idle CPU usage

github.com/sgl-project/sglang - zyearw1024 opened this pull request 3 months ago
[Bug] 100% CPU Usage When Idle in sglang

github.com/sgl-project/sglang - zyearw1024 opened this issue 3 months ago
[Bug][minimal reproducible demo] High variability across batch inference runs

github.com/sgl-project/sglang - FredericOdermatt opened this issue 3 months ago
[LoRA, Performance] Add gemm expand triton kernel for multi-LoRA

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
[Bugfix] qwen2vl forward_extend

github.com/sgl-project/sglang - yizhang2077 opened this pull request 3 months ago
Split the overlapped version of TpModelWorkerClient into a separate file

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Temporarily skip the test_mixed_batch for QWen2VL

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Unify the memory pool api and tp worker API

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
docs: fix README

github.com/sgl-project/sglang - zhyncs opened this pull request 3 months ago
Update README.md

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
Support qwen2 vl model

github.com/sgl-project/sglang - zhyncs opened this pull request 3 months ago
Update vllm to 0.6.3 (#1711)

github.com/sgl-project/sglang - zhyncs opened this pull request 3 months ago
CPU Inference

github.com/sgl-project/sglang - JocelynPanPan opened this issue 3 months ago
Simplify the interface of tp_worker

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Created SECURITY.md

github.com/sgl-project/sglang - NishantRana07 opened this pull request 3 months ago
Update readme and workflow

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Feature] Cascade attention kernels

github.com/sgl-project/sglang - merrymercy opened this issue 3 months ago
Release v0.3.4

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix the race condition in overlap mode

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Update vllm to 0.6.3

github.com/sgl-project/sglang - ispobock opened this pull request 3 months ago
Fix `is_all_ready` for overlap copy

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Simplify the nan detection and greedy check in sampler

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Does frontend language support multi-image QA?

github.com/sgl-project/sglang - joeyy5588 opened this issue 3 months ago
Skip unnecessary penalizer

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add grouped free operations

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add dtype for more operations

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Simplify flashinfer utilities

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix regex and logprob conflicts when chunked prefilling

github.com/sgl-project/sglang - hnyls2002 opened this pull request 3 months ago
Fix mixed batch for multi modal models

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix engine unit test

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix failed ci tests on long prompts; Better error messages for embedding models

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix the failed unit tests

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Bug] AttributeError in `openai.Client` Embeddings API

github.com/sgl-project/sglang - tanzelin430 opened this issue 3 months ago
feat: radix tree code optimize

github.com/sgl-project/sglang - wxsms opened this pull request 3 months ago
Use SGLang imports for linear layer

github.com/sgl-project/sglang - janimo opened this pull request 3 months ago
[Router] Implement router backbone

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
ORJson. Faster Json serialization

github.com/sgl-project/sglang - michaelfeil opened this pull request 3 months ago
[Bug] crash about `c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout`

github.com/sgl-project/sglang - zeng-zc opened this issue 3 months ago
[Bug] IndexError: Inconsistent batch_size and len(image_input)

github.com/sgl-project/sglang - OBJECT907 opened this issue 3 months ago
[Bug] deadlock or hang on Qwen2-7B models

github.com/sgl-project/sglang - zeng-zc opened this issue 3 months ago
Update the transformers version in CI

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
add orjson for jsonresponse

github.com/sgl-project/sglang - michaelfeil opened this pull request 3 months ago
Launch a thread to overlap CPU and GPU

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Event] Add online meetup meeting link

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
Fix srt dependency

github.com/sgl-project/sglang - ispobock opened this pull request 3 months ago
Add matched_stop token or str to distinguish between eos or stop str finish_reason generation

github.com/sgl-project/sglang - g-drozdov opened this pull request 3 months ago
[Bug] ROCm6.1.2 sglang0.3.3 cuda graph coredump

github.com/sgl-project/sglang - linqingxu opened this issue 3 months ago
Fixes for running reward model inference using sglang

github.com/sgl-project/sglang - corbt opened this pull request 3 months ago
Fix filter_batch function call

github.com/sgl-project/sglang - hnyls2002 opened this pull request 3 months ago
[Performance] Support `xgrammar` for faster constrained decoding

github.com/sgl-project/sglang - DarkSharpness opened this pull request 3 months ago
Add date to logging messages (#1623)

github.com/sgl-project/sglang - zeng-zc opened this pull request 3 months ago
slides link to .pdf

github.com/sgl-project/sglang - ziliangpeng opened this pull request 3 months ago
Add a new event loop

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add OLMo model

github.com/sgl-project/sglang - janimo opened this pull request 3 months ago