SGLang issues | Ecosyste.ms: OpenCollective

[Feature] Add a flag for computing the prompt's logprobs or not.

github.com/sgl-project/sglang - hnyls2002 opened this issue 5 months ago

[Bug] 运行sglang.launch_server报错：cannot import name 'default_dump_dir' from 'triton.runtime.cache'

github.com/sgl-project/sglang - NoobPythoner opened this issue 5 months ago

run llama 3.1 405B with multi node has tp server error [Bug]

github.com/sgl-project/sglang - kinglion811 opened this issue 5 months ago

[Bug] AWQ Marlin not work with Torch Compile

github.com/sgl-project/sglang - zhyncs opened this issue 5 months ago

RuntimeError: TopKTopPSamplingFromProbs failed with error code no kernel image is available for execution on the device 已杀死[Bug]

github.com/sgl-project/sglang - mayu123mayu opened this issue 5 months ago

[Feature] plan to support medusa?

github.com/sgl-project/sglang - CSEEduanyu opened this issue 5 months ago

[Bug] Multi-Node communication issue

github.com/sgl-project/sglang - dmakhervaks opened this issue 5 months ago

[Feature] RadixCache: remove recursive logic

github.com/sgl-project/sglang - hnyls2002 opened this issue 5 months ago

OPTIONS method is not supported when using sglang with the nextchat client

github.com/sgl-project/sglang - jjiwei opened this issue 5 months ago

[Feature] Frontend: be able to run generate super long text

github.com/sgl-project/sglang - xianbaoqian opened this issue 5 months ago

ROCM

github.com/sgl-project/sglang - BasDiaz opened this issue 5 months ago

[Feature] Generation Inputs: input_embeds

github.com/sgl-project/sglang - AlekseyKorshuk opened this issue 5 months ago

Initialization failed. warmup error:

github.com/sgl-project/sglang - bravelll opened this issue 5 months ago

Support for WebAssembly models

github.com/sgl-project/sglang - jaanli opened this issue 6 months ago

Development Roadmap (2024 Q3)

github.com/sgl-project/sglang - Ying1123 opened this issue 6 months ago

select() on first assistant token broken (in different ways in Mistral and Llama). Likely tokenization issue.

github.com/sgl-project/sglang - max99x opened this issue 6 months ago

`model_override_args` with server

github.com/sgl-project/sglang - ValeKnappich opened this issue 6 months ago

Add a HuggingFace backend

github.com/sgl-project/sglang - cloneofsimo opened this issue 6 months ago

Function calling for OpenAI backend

github.com/sgl-project/sglang - Yiyun-Liang opened this pull request 6 months ago

Add Support to Florence-2

github.com/sgl-project/sglang - KaifAhmad1 opened this issue 6 months ago

Will speculative decoding be supported?

github.com/sgl-project/sglang - arunpatala opened this issue 7 months ago

Llava CUDA error: device-side assert triggered

github.com/sgl-project/sglang - dmilcevski opened this issue 7 months ago

[Bug]: Random model output using sglang backend server

github.com/sgl-project/sglang - PanJason opened this issue 7 months ago

SG-Lang Runtime Stuck Launching in Docker Container

github.com/sgl-project/sglang - schopra8 opened this issue 7 months ago

Qwen 2 7B not working

github.com/sgl-project/sglang - sudarshan-kamath opened this issue 7 months ago

Does llava-next-video deploy only focus on first frames?

github.com/sgl-project/sglang - LetheRiver0 opened this issue 7 months ago

Unable to load 72b llava qwen on 8*A100 40GB

github.com/sgl-project/sglang - jeffhernandez1995 opened this issue 7 months ago

remove redundant pad_input_ids function

github.com/sgl-project/sglang - amosyou opened this pull request 7 months ago

llava-next-video inference result is empty

github.com/sgl-project/sglang - AmazDeng opened this issue 7 months ago

no longer can load 72b llava qwen on 4*H100 80GB

github.com/sgl-project/sglang - pseudotensor opened this issue 7 months ago

Invalid API key

github.com/sgl-project/sglang - pseudotensor opened this issue 8 months ago

Trace OpenAI backend usage

github.com/sgl-project/sglang - Ying1123 opened this issue 8 months ago

Regex generation causes 37x lower performance

github.com/sgl-project/sglang - Gintasz opened this issue 8 months ago

OOM CUDA error on 8 * L4 machine when launching sglang server

github.com/sgl-project/sglang - mounamokaddem opened this issue 8 months ago

Llama-3 regex generation can get stuck in infinite generation beyond max_tokens and crash server (reproduction example)

github.com/sgl-project/sglang - Gintasz opened this issue 8 months ago

Please add Phi3 support

github.com/sgl-project/sglang - Curiosity007 opened this issue 8 months ago

no batch run when using openai's format for calling.

github.com/sgl-project/sglang - xjw00654 opened this issue 8 months ago

ImportError: cannot import name 'function' from partially initialized module 'sglang'

github.com/sgl-project/sglang - lambda7xx opened this issue 9 months ago

ImportError: cannot import name 'get_cuda_stream' from 'triton.runtime.jit' In triton-nightly(V100)

github.com/sgl-project/sglang - nenomigami opened this issue 9 months ago

Add Default Timeout to urllib.request.urlopen Calls to Prevent Potential Hanging

github.com/sgl-project/sglang - alessiodallapiazza opened this issue 9 months ago

Allow OPTIONS Method on Http Server and add Cors headers.

github.com/sgl-project/sglang - kseyhan opened this issue 10 months ago

Supports the InternVL multimodal large model

github.com/sgl-project/sglang - exceedzhang opened this issue 10 months ago

Openrouter usage example