Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

SGLang

SGLang is a fast serving framework for large language models and vision language models.
Collective - Host: opensource - https://opencollective.com/sglang - Code: https://github.com/sgl-project/sglang

Fix memory leak during abort

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Feature] Make vLLM optional in model code

github.com/sgl-project/sglang - ByronHsu opened this issue 3 months ago
Improve benchmark scripts

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Minor] Add some utility functions

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[doc] improve engine doc and add to readme

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Feature] When will a version of S-Lora be available?

github.com/sgl-project/sglang - kunkunzhang123 opened this issue 3 months ago
Simplify chunked prefill

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Minor] Improve style

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix unit test order to balance the tasks in CI

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Bug] difference of kv-cache-prefixing between vLLM and sglang

github.com/sgl-project/sglang - chenchunhui97 opened this issue 3 months ago
Move filter_batch out of stream_output

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add a test case to test retract

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Minor] Rename no_eos_trim to no_stop_trim

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
docs: add zh_CN po files

github.com/sgl-project/sglang - llama-factory opened this pull request 3 months ago
Add output_ids into ScheduleBatch

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[1/N] Remove `CacheConfig` import in all model files

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
temp

github.com/sgl-project/sglang - yukavio opened this pull request 3 months ago
[doc] Add engine section in backend.md

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Feature] sanic Custom Server example support openai stream api ?

github.com/sgl-project/sglang - lys791227 opened this issue 3 months ago
Fix the batch_is_full check for jump-forward decoding

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add get_tokenizer function for Engine class

github.com/sgl-project/sglang - pjyi2147 opened this pull request 3 months ago
Simplify the event loop and expose `--num-continuous-decode-steps` as an argument

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add an option to disable penalizer

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Fix] fix eos trim inconsistency

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
[Feature] Multi-instance deployment

github.com/sgl-project/sglang - vkc1vk opened this issue 3 months ago
Fix unit tests and type annotations

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
docs: add zh_CN po files

github.com/sgl-project/sglang - llama-factory opened this pull request 3 months ago
dead

github.com/sgl-project/sglang - llama-factory opened this pull request 3 months ago
Fix ignore_eos in the OpenAI ChatCompletions API

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Surpport kv cache int8/int4 for triton backend

github.com/sgl-project/sglang - yuguo-Jack opened this pull request 3 months ago
[Feature] Support for GPT-2

github.com/sgl-project/sglang - hariharan98m opened this issue 3 months ago
fix missing ignore_eos in v1/chat/completions

github.com/sgl-project/sglang - learninmou opened this pull request 3 months ago
Add GLM-4v Multimodal Model support for SGLang

github.com/sgl-project/sglang - sixsixcoder opened this pull request 3 months ago
is it time to rerun the benchmarks?

github.com/sgl-project/sglang - stas00 opened this issue 3 months ago
[Fix] Fix the style of test_large_max_new_tokens.py

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Increase backend-test2 timeout from 20 to 25 mins

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
Release v0.3.3.post1

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add back data parallelism

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix constrained decoding

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Added a "Back To Top" Button

github.com/sgl-project/sglang - JanumalaAkhilendra opened this pull request 3 months ago
Add image_token in conversation.py

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix the correctness test in bench_latency.py when tp > 1 and test_generation_models.py

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Bug] The time unit in bench_serving is wrong on A800-SXM4-40GB, using perf_counter_ns could fix

github.com/sgl-project/sglang - zeng-zc opened this issue 3 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Add the ability to enable and disable the Profiler via HTTP API.

github.com/sgl-project/sglang - abatom opened this pull request 3 months ago
Update README.md

github.com/sgl-project/sglang - kushal34712 opened this pull request 3 months ago
[Feature] Using frontend APIs but passing a list of prompts in `run` rather than `run_batch`

github.com/sgl-project/sglang - pengye91 opened this issue 3 months ago
[Feature] Add date to logging

github.com/sgl-project/sglang - zeng-zc opened this issue 3 months ago
[Bug] JSON Regex does not work for vision model

github.com/sgl-project/sglang - ByronHsu opened this issue 3 months ago
Fix test_vision_openai_server on CI

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Performance, hardware] MoE tuning update to AMD MI300x GPUs

github.com/sgl-project/sglang - HaiShaw opened this pull request 3 months ago
[Question]Where is data parallelism

github.com/sgl-project/sglang - josephydu opened this issue 3 months ago
[Bug] Message to guide using <=0.3.2 for data parallel is not shown when --dp is set

github.com/sgl-project/sglang - seungduk-yanolja opened this issue 3 months ago
[Feature] GGUF support

github.com/sgl-project/sglang - remixer-dec opened this issue 3 months ago
[Minor] Fix logging typo

github.com/sgl-project/sglang - amosyou opened this pull request 3 months ago
[engine] support async and streaming

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Bug] SGLang hangs after hitting 0.00 token usage on Engine.generate

github.com/sgl-project/sglang - pjyi2147 opened this issue 3 months ago
Nit about the decorator of `PortArgs.init_new`

github.com/sgl-project/sglang - glen-amd opened this pull request 3 months ago
[Bug] Can't access one click llms on runpod

github.com/sgl-project/sglang - kovern opened this issue 3 months ago
[Bug] Can't distinguish whether the EOS token or the stop token caused the generation to stop

github.com/sgl-project/sglang - g-drozdov opened this issue 3 months ago
[Feature] Log input text instead of input_ids when using openai chat apis

github.com/sgl-project/sglang - CedricHwong opened this issue 3 months ago
Add device support

github.com/sgl-project/sglang - liangan1 opened this pull request 3 months ago
[Engine] Fix generate hanging issue after the first call

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
Release v0.3.3

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Profile] Add pytorch profiler

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
Remove references to squeezellm

github.com/sgl-project/sglang - janimo opened this pull request 3 months ago
[WIP] Support NVLM-D

github.com/sgl-project/sglang - amosyou opened this pull request 3 months ago
Update README.md

github.com/sgl-project/sglang - kushal34712 opened this pull request 3 months ago
Returning a per request metric for number of cached_tokens read

github.com/sgl-project/sglang - havetc opened this pull request 3 months ago
Optimize broadcast & Reorg code

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix the port_args in bench_latency

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Use is_flashinfer_available to replace is_hip for flashinfer check

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Use `atexit` hook to implicitly shutdown `Runtime`

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
Fix chunked prefill condition

github.com/sgl-project/sglang - ispobock opened this pull request 3 months ago
[Fix] Fix the case where prompt_len = 0

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix modality for image inputs

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Test consistency for single and batch seperately

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Minor, Performance] Use torch.argmax for greedy sampling

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
fix(docs): Improve grammar and readability in README

github.com/sgl-project/sglang - amantyagiprojects opened this pull request 3 months ago
[LoRA, Performance] Speedup multi-LoRA serving - Step 1

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
Clean up event loop

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
[Bug] Fix decode stats error on output_len 1

github.com/sgl-project/sglang - HaiShaw opened this pull request 3 months ago
[Minor] Improve the style and fix flaky tests

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago
Fix styling

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
Fix runtime.generate when sampling param is not passed

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
default sampling param should be deepcopied

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
chore: update README.md

github.com/sgl-project/sglang - eltociear opened this pull request 3 months ago
[Bug] Fix the Image Input of Batch Generation

github.com/sgl-project/sglang - OBJECT907 opened this pull request 3 months ago
Update io_struct.py

github.com/sgl-project/sglang - OBJECT907 opened this pull request 3 months ago
[Easy] use .text() instead of .text

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
Backend method not found when SRT Runtime is used

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Bug] Inconsistent results when executing independent sglang functions in different orders

github.com/sgl-project/sglang - ByronHsu opened this issue 3 months ago
Refine the add request reasons to avoid corner cases.

github.com/sgl-project/sglang - hnyls2002 opened this pull request 3 months ago
Support min_tokens in sgl.gen

github.com/sgl-project/sglang - ByronHsu opened this pull request 3 months ago
[Event] Update README.md

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
[Bug] `Meta-Llama-3.1-8B-Instruct` triggers "Detected errors during sampling! NaN in the probability." under high concurrency/RPS.

github.com/sgl-project/sglang - tongyx361 opened this issue 3 months ago
[LoRA, Performance] Speedup multi-LoRA serving - Step 1

github.com/sgl-project/sglang - Ying1123 opened this pull request 3 months ago
[Minifix] Remove extra space in cot example

github.com/sgl-project/sglang - FredericOdermatt opened this pull request 3 months ago
Make input_ids a torch.Tensor

github.com/sgl-project/sglang - merrymercy opened this pull request 3 months ago