Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

SGLang

SGLang is a fast serving framework for large language models and vision language models.
Collective - Host: opensource - https://opencollective.com/sglang - Code: https://github.com/sgl-project/sglang

Use `atexit` hook to implicitly shutdown `Runtime`

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Fix chunked prefill condition

github.com/sgl-project/sglang - ispobock opened this pull request 4 months ago
[Fix] Fix the case where prompt_len = 0

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix modality for image inputs

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Test consistency for single and batch seperately

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Minor, Performance] Use torch.argmax for greedy sampling

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
fix(docs): Improve grammar and readability in README

github.com/sgl-project/sglang - amantyagiprojects opened this pull request 4 months ago
[LoRA, Performance] Speedup multi-LoRA serving - Step 1

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Clean up event loop

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] Fix decode stats error on output_len 1

github.com/sgl-project/sglang - HaiShaw opened this pull request 4 months ago
[Minor] Improve the style and fix flaky tests

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix styling

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Fix runtime.generate when sampling param is not passed

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
default sampling param should be deepcopied

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
chore: update README.md

github.com/sgl-project/sglang - eltociear opened this pull request 4 months ago
[Bug] Fix the Image Input of Batch Generation

github.com/sgl-project/sglang - OBJECT907 opened this pull request 4 months ago
Update io_struct.py

github.com/sgl-project/sglang - OBJECT907 opened this pull request 4 months ago
[Easy] use .text() instead of .text

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Backend method not found when SRT Runtime is used

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Bug] Inconsistent results when executing independent sglang functions in different orders

github.com/sgl-project/sglang - ByronHsu opened this issue 4 months ago
Refine the add request reasons to avoid corner cases.

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Support min_tokens in sgl.gen

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Event] Update README.md

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
[Bug] `Meta-Llama-3.1-8B-Instruct` triggers "Detected errors during sampling! NaN in the probability." under high concurrency/RPS.

github.com/sgl-project/sglang - tongyx361 opened this issue 4 months ago
[LoRA, Performance] Speedup multi-LoRA serving - Step 1

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
[Minifix] Remove extra space in cot example

github.com/sgl-project/sglang - FredericOdermatt opened this pull request 4 months ago
Make input_ids a torch.Tensor

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Provide an offline engine API

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Use ipc instead of tcp in zmq

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[doc] Chinese Documentation Translation Available for sglang

github.com/sgl-project/sglang - khum08 opened this issue 4 months ago
[Feature] Add `choices` in `/generate` endpoint and add `min_new_tokens` in `sgl.gen()`

github.com/sgl-project/sglang - TING2938 opened this issue 4 months ago
[Fix] Fix major performance bug in certain cases

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Organize sampling batch info better

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add llama implementation with no tensor parallel linears

github.com/sgl-project/sglang - jerryzh168 opened this pull request 4 months ago
Print out what the model saw?

github.com/sgl-project/sglang - cinjon opened this issue 4 months ago
[FP8 KV Cache] Avoid KeyError at loading pre-quantized FP8 model with kv_scale

github.com/sgl-project/sglang - HaiShaw opened this pull request 4 months ago
[Bug] Exception: Capture cuda graph failed: Triton Error [CUDA]: device kernel image is invalid

github.com/sgl-project/sglang - a136214808 opened this issue 4 months ago
Move status check in the memory pool to CPU

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Fix] Move ScheduleBatch out of SamplingInfo

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Fix] do not maintain regex_fsm in SamplingBatchInfo

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Performance, Hardware] MoE tuning on AMD MI300x GPUs

github.com/sgl-project/sglang - kkHuang-amd opened this pull request 4 months ago
[Fix] Fix all the Huggingface paths

github.com/sgl-project/sglang - tbarton16 opened this pull request 4 months ago
Simplify flashinfer dispatch

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Llama3.2 vision model support

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Dispatch flashinfer wrappers

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
[Refactor] Simplify io_struct and tokenizer_manager

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Fix bugs of `logprobs_nums`

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Organize Attention Backends

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Support qwen2 vl model

github.com/sgl-project/sglang - yizhang2077 opened this pull request 5 months ago
[Fix, LoRA] fix LoRA with updates in main

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
Clean up batch data structures: Introducing ModelWorkerBatch

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
Rename InputMetadata -> ForwardBatch

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
Add support for Molmo-D-7B Model

github.com/sgl-project/sglang - BabyChouSr opened this pull request 5 months ago
Let ModelRunner take InputMetadata as input, instead of ScheduleBatch

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
[Refactor] Simplify io_struct and tokenizer_manager

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
Process image in parallel

github.com/sgl-project/sglang - hnyls2002 opened this pull request 5 months ago
Move scheduler code from tp_worker.py to scheduler.py

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
fix ipv6 url when warm up model

github.com/sgl-project/sglang - cauyxy opened this pull request 5 months ago
[Fix] Fix AttributeError in Qwen2.5 LoRA: 'Qwen2ForCausalLM' object has no attribute 'get_hidden_dim'

github.com/sgl-project/sglang - mssongit opened this pull request 5 months ago
[Fix] Fix AttributeError in Qwen2.5(huggingface model) LoRA: 'Qwen2ForCausalLM' object has no attribute 'get_module_name'

github.com/sgl-project/sglang - mssongit opened this pull request 5 months ago
Improve process creation

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
[Bug] ValueError: The memory capacity is unbalanced

github.com/sgl-project/sglang - chuangzhidan opened this issue 5 months ago
Make detokenizer_manager.py not asyncio

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
Organize image inputs

github.com/sgl-project/sglang - hnyls2002 opened this pull request 5 months ago
Multiple minor fixes

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
[Event] Update meeting link

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
Add float8 dynamic quant to torchao_utils

github.com/sgl-project/sglang - jerryzh168 opened this pull request 5 months ago
[Feature] VLLM 6.0 support

github.com/sgl-project/sglang - arunpatala opened this issue 5 months ago
[Bug] IndexError: list index out of range

github.com/sgl-project/sglang - lvxianfeng-git opened this issue 5 months ago
[Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
minor: fix config

github.com/sgl-project/sglang - hnyls2002 opened this pull request 5 months ago
[Feature] add support for llama 3.2

github.com/sgl-project/sglang - Stealthwriter opened this issue 5 months ago
[Bug] Unable to use gptq or awq with torch.compile (8*A40)

github.com/sgl-project/sglang - smallstepman opened this issue 5 months ago
[FIX] Catch syntax error of Regex Guide to avoid crash

github.com/sgl-project/sglang - du00cs opened this pull request 5 months ago
[bugfix]Add modelscope package to avoid docker image without modelscope

github.com/sgl-project/sglang - KylinMountain opened this pull request 5 months ago
Accuracy reduction of Lora

github.com/sgl-project/sglang - yileld opened this issue 5 months ago
Update Dockerfile

github.com/sgl-project/sglang - KylinMountain opened this pull request 5 months ago
[Bug] no module modelscope using docker compose to start sglang

github.com/sgl-project/sglang - KylinMountain opened this issue 5 months ago
How to study the code?

github.com/sgl-project/sglang - TJ949 opened this issue 5 months ago
[Feature] _get_pixel_values needs to return tgt_sizes

github.com/sgl-project/sglang - huangzl18883 opened this issue 5 months ago
[Fix] Ignore model import error

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
Release v0.3.2

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
Revert "kernel: use tensor cores for flashinfer gqa kernels"

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
[Fix] Fix clean_up_tokenization_spaces in tokenizer

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
[Bug] tensor parallel run error

github.com/sgl-project/sglang - jerryzh168 opened this issue 5 months ago
Add support for tie_word_embeddings when loading weights + support for SmolLM

github.com/sgl-project/sglang - TianyiQ opened this pull request 5 months ago
[CI] Update nightly eval

github.com/sgl-project/sglang - Ying1123 opened this pull request 5 months ago
[Bug] LLaVa-next does not work for single image processing

github.com/sgl-project/sglang - ThomasBenzshawel opened this issue 5 months ago
AWQ performance tracking

github.com/sgl-project/sglang - zhyncs opened this issue 5 months ago
Possible timing side-channels caused by shared prefix

github.com/sgl-project/sglang - Unik-lif opened this issue 5 months ago
Simplify bench_latency.py

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
Update test_srt_backend.py

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
[Bug] radixcache stack_overflow

github.com/sgl-project/sglang - luzengxiangcn opened this issue 5 months ago
[CI] Move AMD test to a separate file

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
debug radixcache stack_overflow

github.com/sgl-project/sglang - luzengxiangcn opened this pull request 5 months ago
Speculative decoding with EAGLE2

github.com/sgl-project/sglang - yukavio opened this pull request 5 months ago
MoE torch compile

github.com/sgl-project/sglang - ispobock opened this pull request 5 months ago
Fix the overhead due to penalizer in bench_latency

github.com/sgl-project/sglang - merrymercy opened this pull request 5 months ago
Fix RuntimeEndpoint.select method

github.com/sgl-project/sglang - jeffrey-fong opened this pull request 5 months ago