Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

SGLang

SGLang is a fast serving framework for large language models and vision language models.
Collective - Host: opensource - https://opencollective.com/sglang - Code: https://github.com/sgl-project/sglang

[CI] Return output logprobs in unit test

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Unify forward mode

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
[Feature] Follow up on non power of 2 triton kernel

github.com/sgl-project/sglang - ByronHsu opened this issue 4 months ago
[Bug] it seems memory leak in sglang when longtime serving

github.com/sgl-project/sglang - CSEEduanyu opened this issue 4 months ago
[Minor] Many cleanup

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Feature] support LLaVA-NeXT-Video-32B-Qwen

github.com/sgl-project/sglang - HarperGG opened this issue 4 months ago
[Feature] smooth quant or other quant method

github.com/sgl-project/sglang - MichoChan opened this issue 4 months ago
[Feature] support qwen2 vl

github.com/sgl-project/sglang - zhyncs opened this issue 4 months ago
[Feature] KV Cache Quantization

github.com/sgl-project/sglang - ghost opened this issue 4 months ago
[Feature] DRY repetition penalty

github.com/sgl-project/sglang - vnkc1 opened this issue 4 months ago
[Bug] `served_model_name` argument in the server_arg.py is not checked

github.com/sgl-project/sglang - zhaochenyang20 opened this issue 4 months ago
[Feature] KV Cache Compression

github.com/sgl-project/sglang - ghost opened this issue 4 months ago
[Feat] Add modalities for vision server when handling pixel values for llava

github.com/sgl-project/sglang - kcz358 opened this pull request 4 months ago
Fix some online scheduling delay

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
[Bug] it didn't work when using tp on RTX 3090

github.com/sgl-project/sglang - milktea888 opened this issue 4 months ago
jinja2.exceptions.TemplateError: System role not supported

github.com/sgl-project/sglang - sdecoder opened this issue 4 months ago
Add torchao quant (int4/int8/fp8) to llama models

github.com/sgl-project/sglang - jerryzh168 opened this pull request 4 months ago
docs: add conclusion

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
Optimize schedule

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
[Bug] Multi machine, multi card, slow speed

github.com/sgl-project/sglang - guleng opened this issue 4 months ago
docs: highlight ttft itl and throughput

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
docs: update README

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
[Feature] Per-request random seed

github.com/sgl-project/sglang - laoconeth opened this issue 4 months ago
[Bug] ConnectionResetError: [Errno 104] Connection reset by peer

github.com/sgl-project/sglang - oliver-li opened this issue 4 months ago
[Bug] Unsupported architectures: ChatGLMForConditionalGeneration.

github.com/sgl-project/sglang - maxin9966 opened this issue 4 months ago
[Bug] Using 8 H20 GPUs, the deepseek-coder-v2-fp8 starts up normally, but there is no response to client requests.

github.com/sgl-project/sglang - fengyang95 opened this issue 4 months ago
Remove useless fields in global_config.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
docs: update news

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
Fix the flaky test test_moe_eval_accuracy_large.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] T4 Crash

github.com/sgl-project/sglang - Abdulhanan535 opened this issue 4 months ago
[Bug] RuntimeError in ModelTpServer

github.com/sgl-project/sglang - Lzhang-hub opened this issue 4 months ago
[Feature] support smooth-quant?

github.com/sgl-project/sglang - Lzhang-hub opened this issue 4 months ago
[Bug] Facing Error When starting.

github.com/sgl-project/sglang - Abdulhanan535 opened this issue 4 months ago
chore: bump v0.3.0

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
misc: speedup load safetensors

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
Fix select by ensuring each request has at least one token

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
Fix llama2 weight loader

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] Unable to fix model output

github.com/sgl-project/sglang - cherishhh opened this issue 4 months ago
The CPU is also occupied at 100% when there are no requests.

github.com/sgl-project/sglang - luhairong11 opened this issue 4 months ago
Update README.md for llava-onevision instructions

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] gen with regex: Token fusion between input and output, try to avoid this by removing the space at the end of the input.

github.com/sgl-project/sglang - alanxmay opened this issue 4 months ago
Removed unused methods

github.com/sgl-project/sglang - janimo opened this pull request 4 months ago
[Bug] Update to 0.2.15 and torch compile leads to error

github.com/sgl-project/sglang - zhaochenyang20 opened this issue 4 months ago
Adding document for backend

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 4 months ago
[Fix] Reduce memory usage for loading llava model & Remove EntryClassRemapping

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Feature] Initial support for multi-LoRA serving

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Fix bugs in sampler with CUDA graph / torch.compile

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
feat: update linear deps 1/N

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
feat: update nightly gsm8k eval

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
Do you support frontend-language inference for Llava-OneVision ?

github.com/sgl-project/sglang - ehayeshaiper opened this issue 4 months ago
[Bug] A100 PCIE torch compile error

github.com/sgl-project/sglang - zhyncs opened this issue 4 months ago
Adding Documentation for installation

github.com/sgl-project/sglang - zhaochenyang20 opened this pull request 4 months ago
Support Phi3 mini and medium

github.com/sgl-project/sglang - janimo opened this pull request 4 months ago
[server] Passing `model_override_args` to `launch_server` via the CLI.

github.com/sgl-project/sglang - kevin85421 opened this pull request 4 months ago
Fix hang when doing s += None.

github.com/sgl-project/sglang - max99x opened this pull request 4 months ago
Fix regex mask

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Release v0.2.15

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[doc] Fix more broken links

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Fix the flaky tests in test_moe_eval_accuracy_large.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Feature] Correctness test for Triton kernels

github.com/sgl-project/sglang - ByronHsu opened this issue 4 months ago
ci: add nightly eval

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
fix: resolve fp8 for mixtral

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
[CI] merge all ci tests into one file

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[triton] Remove the zero initialization of qk_acc by directly writing the result

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
Separated control and compute loop, shorten the critical path, and enable more complicated policies

github.com/sgl-project/sglang - Ying1123 opened this pull request 4 months ago
Support Triton fp8 e5m2 kv cache

github.com/sgl-project/sglang - ispobock opened this pull request 4 months ago
feat: fix fp8 for MLA and support bmm fp8 for DeepSeek V2

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
[Chore] Rename model_overide_args to model_override_args

github.com/sgl-project/sglang - kevin85421 opened this pull request 4 months ago
[Feature] Support phi-3 model

github.com/sgl-project/sglang - ByronHsu opened this issue 4 months ago
[doc] fix quick start link

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[triton] Support head_dim not 2^n in triton extend and decode attention

github.com/sgl-project/sglang - ByronHsu opened this pull request 4 months ago
[CI] Add more multi-gpu tests

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] device-side assert triggered when using run_batch

github.com/sgl-project/sglang - stikkireddy opened this issue 4 months ago
Optimize new token calculation

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
Allow new lines during JSON generation

github.com/sgl-project/sglang - qeternity opened this pull request 4 months ago
fix: resolve the fp8 bug introduced by vLLM 0.5.5

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
[Bug] sglang.launch_server error

github.com/sgl-project/sglang - andyluo7 opened this issue 4 months ago
[Bug] Device-side assert triggered in logits processor when running Llama 3.1 70B

github.com/sgl-project/sglang - hrukalive opened this issue 4 months ago
[Feature] support long context eval and benchmark

github.com/sgl-project/sglang - zhyncs opened this issue 4 months ago
[Feature] support nightly eval

github.com/sgl-project/sglang - zhyncs opened this issue 4 months ago
[Feature] support ultravox

github.com/sgl-project/sglang - zhyncs opened this issue 4 months ago
[Bug] sglang run for few hours, it will stop returning valid response

github.com/sgl-project/sglang - liho00 opened this issue 4 months ago
Report median instead of mean in bench_latency.py

github.com/sgl-project/sglang - merrymercy opened this pull request 4 months ago
[Bug] Why sglang is slower than vllm on ShareGPT datasets?

github.com/sgl-project/sglang - lullabies777 opened this issue 4 months ago
Update README Support Exaone 3.0

github.com/sgl-project/sglang - Deepfocused opened this pull request 4 months ago
[Bug] OpenAI Compatible Prompt Template Error

github.com/sgl-project/sglang - BabyChouSr opened this issue 4 months ago
[Bug] Lower single request speed with mla enabled

github.com/sgl-project/sglang - halexan opened this issue 4 months ago
Optimize the update flashinfer indices

github.com/sgl-project/sglang - xiaobochen123 opened this pull request 4 months ago
Transpose mla weight offline

github.com/sgl-project/sglang - ispobock opened this pull request 4 months ago
fix: multimodal_config in monkey_patch_vllm_dummy_weight_loader

github.com/sgl-project/sglang - lxww302 opened this pull request 4 months ago
[Bug] cannot set --load-format=dummy with vllm 0.5.5

github.com/sgl-project/sglang - lxww302 opened this issue 4 months ago
EXAONE 3.0 Model Support

github.com/sgl-project/sglang - Deepfocused opened this pull request 4 months ago
[Bug] incorrect input_tokens_logprob slicing in RuntimeEndpoint.select method

github.com/sgl-project/sglang - jeffrey-fong opened this issue 4 months ago
Allow more flexible assistant and system response

github.com/sgl-project/sglang - BabyChouSr opened this pull request 4 months ago
fix data racing due to mutable reference using deepcopy

github.com/sgl-project/sglang - xiezhq-hermann opened this pull request 4 months ago
make json_schema usable from gen

github.com/sgl-project/sglang - qeternity opened this pull request 4 months ago
Sampler cudagraph

github.com/sgl-project/sglang - hnyls2002 opened this pull request 4 months ago
fix: resolve qwen2 moe weight loader

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago
[Bug] Error in loading Qwen2-57B-A14B-Instruct

github.com/sgl-project/sglang - LucienShui opened this issue 4 months ago
chore: bump v0.2.14.post2

github.com/sgl-project/sglang - zhyncs opened this pull request 4 months ago