Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

SGLang

SGLang is a fast serving framework for large language models and vision language models.
Collective - Host: opensource - https://opencollective.com/sglang - Code: https://github.com/sgl-project/sglang

Fix regex and logprob conflicts when chunked prefilling

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Fix mixed batch for multi modal models

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix engine unit test

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix failed ci tests on long prompts; Better error messages for embedding models

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix the failed unit tests

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] AttributeError in `openai.Client` Embeddings API

github.com/sgl-project/sglang - tanzelin430 opened this issue 4 months ago
feat: radix tree code optimize

github.com/sgl-project/sglang - wxsms opened this pull request 4 months ago
Use SGLang imports for linear layer

github.com/sgl-project/sglang - janimo opened this pull request 4 months ago
[Router] Implement router backbone

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
ORJson. Faster Json serialization

github.com/sgl-project/sglang - michaelfeil opened this pull request 4 months ago
[Bug] crash about `c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout`

github.com/sgl-project/sglang - zeng-zc opened this issue 4 months ago
[Bug] IndexError: Inconsistent batch_size and len(image_input)

github.com/sgl-project/sglang - OBJECT907 opened this issue 4 months ago
[Bug] deadlock or hang on Qwen2-7B models

github.com/sgl-project/sglang - zeng-zc opened this issue 4 months ago
Update the transformers version in CI

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
add orjson for jsonresponse

github.com/sgl-project/sglang - michaelfeil opened this pull request 4 months ago
Launch a thread to overlap CPU and GPU

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Event] Add online meetup meeting link

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Fix srt dependency

github.com/sgl-project/sglang - ispobock opened this pull request 4 months ago
Add matched_stop token or str to distinguish between eos or stop str finish_reason generation

github.com/sgl-project/sglang - g-drozdov opened this pull request 4 months ago
[Bug] ROCm6.1.2 sglang0.3.3 cuda graph coredump

github.com/sgl-project/sglang - linqingxu opened this issue 4 months ago
Fixes for running reward model inference using sglang

github.com/sgl-project/sglang - corbt opened this pull request 4 months ago
Fix filter_batch function call

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
[Performance] Support `xgrammar` for faster constrained decoding

github.com/sgl-project/sglang - DarkSharpness opened this pull request 4 months ago
Add date to logging messages (#1623)

github.com/sgl-project/sglang - zeng-zc opened this pull request 4 months ago
slides link to .pdf

github.com/sgl-project/sglang - ziliangpeng opened this pull request 4 months ago
Add a new event loop

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add OLMo model

github.com/sgl-project/sglang - janimo opened this pull request 4 months ago
Fix memory leak during abort

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Feature] Make vLLM optional in model code

github.com/sgl-project/sglang - ByronHsu opened this issue 4 months ago
Improve benchmark scripts

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Minor] Add some utility functions

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[doc] improve engine doc and add to readme

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Feature] When will a version of S-Lora be available?

github.com/sgl-project/sglang - kunkunzhang123 opened this issue 4 months ago
Simplify chunked prefill

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Minor] Improve style

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix unit test order to balance the tasks in CI

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] difference of kv-cache-prefixing between vLLM and sglang

github.com/sgl-project/sglang - chenchunhui97 opened this issue 4 months ago
Move filter_batch out of stream_output

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add a test case to test retract

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Minor] Rename no_eos_trim to no_stop_trim

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
docs: add zh_CN po files

github.com/sgl-project/sglang - llama-factory opened this pull request 4 months ago
Add output_ids into ScheduleBatch

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[1/N] Remove `CacheConfig` import in all model files

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
temp

github.com/sgl-project/sglang - yukavio opened this pull request 4 months ago
[doc] Add engine section in backend.md

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Feature] sanic Custom Server example support openai stream api ?

github.com/sgl-project/sglang - lys791227 opened this issue 4 months ago
Fix the batch_is_full check for jump-forward decoding

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add get_tokenizer function for Engine class

github.com/sgl-project/sglang - pjyi2147 opened this pull request 4 months ago
Simplify the event loop and expose `--num-continuous-decode-steps` as an argument

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add an option to disable penalizer

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Fix] fix eos trim inconsistency

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
[Feature] Multi-instance deployment

github.com/sgl-project/sglang - vkc1vk opened this issue 4 months ago
Fix unit tests and type annotations

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
docs: add zh_CN po files

github.com/sgl-project/sglang - llama-factory opened this pull request 4 months ago
dead

github.com/sgl-project/sglang - llama-factory opened this pull request 4 months ago
Fix ignore_eos in the OpenAI ChatCompletions API

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Surpport kv cache int8/int4 for triton backend

github.com/sgl-project/sglang - yuguo-Jack opened this pull request 4 months ago
[Feature] Support for GPT-2

github.com/sgl-project/sglang - hariharan98m opened this issue 4 months ago
fix missing ignore_eos in v1/chat/completions

github.com/sgl-project/sglang - learninmou opened this pull request 4 months ago
Add GLM-4v Multimodal Model support for SGLang

github.com/sgl-project/sglang - sixsixcoder opened this pull request 4 months ago
is it time to rerun the benchmarks?

github.com/sgl-project/sglang - stas00 opened this issue 4 months ago
[Fix] Fix the style of test_large_max_new_tokens.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Increase backend-test2 timeout from 20 to 25 mins

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Release v0.3.3.post1

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add back data parallelism

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix constrained decoding

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Added a "Back To Top" Button

github.com/sgl-project/sglang - JanumalaAkhilendra opened this pull request 4 months ago
Add image_token in conversation.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix the correctness test in bench_latency.py when tp > 1 and test_generation_models.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] The time unit in bench_serving is wrong on A800-SXM4-40GB, using perf_counter_ns could fix

github.com/sgl-project/sglang - zeng-zc opened this issue 4 months ago
Update README.md

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Add the ability to enable and disable the Profiler via HTTP API.

github.com/sgl-project/sglang - abatom opened this pull request 4 months ago
Update README.md

github.com/sgl-project/sglang - kushal34712 opened this pull request 4 months ago
[Feature] Using frontend APIs but passing a list of prompts in `run` rather than `run_batch`

github.com/sgl-project/sglang - pengye91 opened this issue 4 months ago
[Feature] Add date to logging

github.com/sgl-project/sglang - zeng-zc opened this issue 4 months ago
[Bug] JSON Regex does not work for vision model

github.com/sgl-project/sglang - ByronHsu opened this issue 4 months ago
Fix test_vision_openai_server on CI

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Performance, hardware] MoE tuning update to AMD MI300x GPUs

github.com/sgl-project/sglang - HaiShaw opened this pull request 4 months ago
[Question]Where is data parallelism

github.com/sgl-project/sglang - josephydu opened this issue 4 months ago
[Bug] Message to guide using <=0.3.2 for data parallel is not shown when --dp is set

github.com/sgl-project/sglang - seungduk-yanolja opened this issue 4 months ago
[Feature] GGUF support

github.com/sgl-project/sglang - remixer-dec opened this issue 4 months ago
[Minor] Fix logging typo

github.com/sgl-project/sglang - amosyou opened this pull request 4 months ago
[engine] support async and streaming

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[Bug] SGLang hangs after hitting 0.00 token usage on Engine.generate

github.com/sgl-project/sglang - pjyi2147 opened this issue 4 months ago
Nit about the decorator of `PortArgs.init_new`

github.com/sgl-project/sglang - glen-amd opened this pull request 4 months ago
[Bug] Can't access one click llms on runpod

github.com/sgl-project/sglang - kovern opened this issue 4 months ago
[Bug] Can't distinguish whether the EOS token or the stop token caused the generation to stop

github.com/sgl-project/sglang - g-drozdov opened this issue 4 months ago
[Feature] Log input text instead of input_ids when using openai chat apis

github.com/sgl-project/sglang - CedricHwong opened this issue 4 months ago
Add device support

github.com/sgl-project/sglang - liangan1 opened this pull request 4 months ago
[Engine] Fix generate hanging issue after the first call

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Release v0.3.3

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Profile] Add pytorch profiler

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Remove references to squeezellm

github.com/sgl-project/sglang - janimo opened this pull request 4 months ago
[WIP] Support NVLM-D

github.com/sgl-project/sglang - amosyou opened this pull request 4 months ago
Update README.md

github.com/sgl-project/sglang - kushal34712 opened this pull request 4 months ago
Returning a per request metric for number of cached_tokens read

github.com/sgl-project/sglang - havetc opened this pull request 4 months ago
Optimize broadcast & Reorg code

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix the port_args in bench_latency

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Use is_flashinfer_available to replace is_hip for flashinfer check

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago