github.com/sgl-project/sglang issues | Ecosyste.ms: OpenCollective

select() on first assistant token broken (in different ways in Mistral and Llama). Likely tokenization issue.

max99x opened this issue 7 months ago

`model_override_args` with server

ValeKnappich opened this issue 7 months ago

Add a HuggingFace backend

cloneofsimo opened this issue 7 months ago

Function calling for OpenAI backend

Yiyun-Liang opened this pull request 7 months ago

Add Support to Florence-2

KaifAhmad1 opened this issue 7 months ago

Will speculative decoding be supported?

arunpatala opened this issue 7 months ago

Llava CUDA error: device-side assert triggered

dmilcevski opened this issue 8 months ago

[Bug]: Random model output using sglang backend server

PanJason opened this issue 8 months ago

SG-Lang Runtime Stuck Launching in Docker Container

schopra8 opened this issue 8 months ago

Qwen 2 7B not working

sudarshan-kamath opened this issue 8 months ago

Does llava-next-video deploy only focus on first frames?

LetheRiver0 opened this issue 8 months ago

Unable to load 72b llava qwen on 8*A100 40GB

jeffhernandez1995 opened this issue 8 months ago

remove redundant pad_input_ids function

amosyou opened this pull request 8 months ago

llava-next-video inference result is empty

AmazDeng opened this issue 8 months ago

no longer can load 72b llava qwen on 4*H100 80GB

pseudotensor opened this issue 8 months ago

Invalid API key

pseudotensor opened this issue 8 months ago

Trace OpenAI backend usage

Ying1123 opened this issue 8 months ago

Regex generation causes 37x lower performance

Gintasz opened this issue 8 months ago

OOM CUDA error on 8 * L4 machine when launching sglang server

mounamokaddem opened this issue 9 months ago

Llama-3 regex generation can get stuck in infinite generation beyond max_tokens and crash server (reproduction example)

Gintasz opened this issue 9 months ago

Please add Phi3 support

Curiosity007 opened this issue 9 months ago

no batch run when using openai's format for calling.

xjw00654 opened this issue 9 months ago

ImportError: cannot import name 'function' from partially initialized module 'sglang'

lambda7xx opened this issue 9 months ago

ImportError: cannot import name 'get_cuda_stream' from 'triton.runtime.jit' In triton-nightly(V100)

nenomigami opened this issue 9 months ago

Add Default Timeout to urllib.request.urlopen Calls to Prevent Potential Hanging

alessiodallapiazza opened this issue 10 months ago

Allow OPTIONS Method on Http Server and add Cors headers.

kseyhan opened this issue 10 months ago

Supports the InternVL multimodal large model

exceedzhang opened this issue 10 months ago

Openrouter usage example

janimo opened this pull request 10 months ago

Update dependencies

janimo opened this pull request 10 months ago

Setting Data Type from the CLI interface

Reichenbachian opened this issue 10 months ago

Add OpenRouter backend

janimo opened this pull request 11 months ago

Use Anthropic messages API

janimo opened this pull request 11 months ago

Add StableLM model.

janimo opened this pull request 11 months ago

Add logo

merrymercy opened this pull request 11 months ago

RuntimeError: CUDA error: device-side assert triggered when running

aliencaocao opened this issue 11 months ago

Error: Connection Refused by SGLANG Backend

deadpipe opened this issue 11 months ago

Will qwen-vl be supported in the future?

zhangqingwu opened this issue 11 months ago

Does sglang support multi-node backend model?

Luodian opened this issue 11 months ago

Add SGLang usage examples

Ying1123 opened this issue 12 months ago

Development Roadmap (Deprecated)

Ying1123 opened this issue 12 months ago

`RecursionError: maximum recursion depth exceeded while calling a Python object` when inferencing with long input

Ja1Zhou opened this issue 12 months ago

[Bug] liuhaotian/llava-v1.6-mistral-7b doesn't load

fozziethebeat opened this issue 12 months ago

Tutorial for Batch Decoding and Obtaining Log Probs

aflah02 opened this issue about 1 year ago

Can SGL generate list of json?

CSWellesSun opened this issue about 1 year ago

Triton support

TheodoreGalanos opened this issue about 1 year ago

Colab?

srush opened this issue about 1 year ago