github.com/lm-sys/FastChat issues | Ecosyste.ms: OpenCollective

Support LoRA adapters for vLLM worker

andresd95 opened this issue 10 months ago

ChatGLM3-6b Compatibility is expected as soon as possible.

7010G opened this issue 10 months ago

AttributeError: 'Encoding' object has no attribute 'num_tokens'

631068264 opened this issue 10 months ago

stop_str not working for vllm_worker

congchan opened this issue 10 months ago

urllib3.exceptions.newconnectionerror

Yinglibo123 opened this issue 10 months ago

TypeError: Object of type Tensor is not JSON serializable

c-steve-wang opened this issue 10 months ago

Bugfix: Fix issues with streaming

verena-hallitschke opened this pull request 10 months ago

Why don't I get any response when I execute python3 -m fastchat.serve.cli --model-path my model path?

DAAworld opened this issue 10 months ago

Support `--chat-template` for vLLM model worker

thiner opened this issue 10 months ago

Add yandexgpt

thefacetakt opened this pull request 10 months ago

Fix mismatched output in web server

sthemeow opened this pull request 10 months ago

Gradio web server shows error

sthemeow opened this issue 10 months ago

Compatibility Issue with Prefix-Tuned Models using PEFT in FastChat

fzalkow opened this issue 10 months ago

Feature: add guidance support for fastchat

linkedlist771 opened this pull request 10 months ago

Support for cosmosage

surak opened this pull request 10 months ago

API call to export chat logs for the server as a whole or a specific workspace.

mark3apps opened this issue 10 months ago

testing google colab example in repo

silvacarl2 opened this issue 10 months ago

Add Orion-14B template and adapter

renillhuang opened this pull request 10 months ago

Why is controller's heart beat thread daemon set to False?

MichelDucartier opened this issue 10 months ago

update bt code

infwinston opened this pull request 10 months ago

Invoke Google PaLM2 models

sekhar-hari opened this issue 10 months ago

Add basic support of mamba-chat

wln20 opened this pull request 10 months ago

PII Filtering

lisadunlap opened this pull request 10 months ago

add system message support for train.py/train_yuan2.py

Force1ess opened this pull request 10 months ago

add system message support for train.py/train_yuan2.py

Force1ess opened this pull request 10 months ago

fastapi and gradio version

congchan opened this issue 10 months ago

[WIP] Data PII filtering

lisadunlap opened this pull request 11 months ago

Unable to run worker for StripedHyena-Nous-7B

kenneth-miura opened this issue 11 months ago

OpenAI key relay

enochlev opened this issue 11 months ago

Add Gemma

BabyChouSr opened this pull request 11 months ago

In order to support the development of agents, I hope that officials can add support for function roles to dialogue templates.

shell-nlp opened this issue 11 months ago

Support CogVLM & Qwen-VL

Lanture1064 opened this issue 11 months ago

support https://huggingface.co/google/gemma-7b & gemma-2b

yanjian1978 opened this issue 11 months ago

When I call the api '/v1/chat/completions' of API Server, it response incomplete results, but vllm's api response complete results

coreyho opened this issue 11 months ago

vllm api will randomly generate answers slowly when applying asynchronous parallel calls

qy1026 opened this issue 11 months ago

Add Atom-7B-Chat model support

Rayrtfr opened this pull request 11 months ago

Fix solve the problem of log writing conflict when starting multiple …

dafu-wu opened this pull request 11 months ago

Pydantic version conflict with vLLM

thiner opened this issue 11 months ago

Fix incompatible with vllm 0.3.0 and fix size mismatch when fine-tuning model is loading

BRM10213 opened this pull request 11 months ago

fix: pydantic warning

DominguesAddem1974 opened this pull request 11 months ago

TypeError: `dumps_kwargs` keyword arguments are no longer supported. at openai_api_server.py

yanjian1978 opened this issue 11 months ago

Fix None Return Issue in get_prompt and Remove Debug Prints from gen_model_answer.py

nejumi opened this pull request 11 months ago

Replace Elo rating with a rating system of higher predictive power

p-e-w opened this issue 11 months ago

Update Model Description

gebriell opened this issue 11 months ago

fix: learn the stop tokens when training.

congchan opened this pull request 11 months ago

[Question] What does the "95% CI" means on the Leaderboard?

AtomicVar opened this issue 11 months ago

add example finetune package version and train_with_template doc

Oscarjia opened this pull request 11 months ago

How to use multiple Ascend NPUs?

litmonk opened this issue 11 months ago

add example finetune package version and add train_with_template docs

Oscarjia opened this pull request 11 months ago

num_input_tokens_seen value is set to 0

Oscarjia opened this issue 11 months ago

Update conversation.py for deepseek-coder when forcefully setting the system message, it does not work

jianwang-ntu opened this pull request 11 months ago

Update conversation.py for deepseek-coder when forcefully setting the system message, it does not work

jianwang-ntu opened this pull request 11 months ago

Using train_with_template on mistral end up in a model with a loop

christobill opened this issue 11 months ago

Train with system prompt

christobill opened this issue 11 months ago

Wrong reference answer in MT-Bench Question 104

charlie963258 opened this issue 11 months ago

--attn-implementation for Huggingface models to allow for flash-attention-2

ShukantPal opened this pull request 11 months ago

Update arena.md to fix link

logankilpatrick opened this pull request 11 months ago

Aya-101 Killing VLLM

surak opened this issue 11 months ago

AYA-101 killing SGLang

surak opened this issue 11 months ago

ibmmodel

ysuklee opened this pull request 11 months ago

How to decipher the log file?

mikeda100 opened this issue 11 months ago

How to save conversation history in a nice-looking way?

mikeda100 opened this issue 11 months ago

Fix SGLang worker

merrymercy opened this pull request 11 months ago

Multi model on single GPU ?

aalev opened this issue 11 months ago

open_api_sever can't stream answer when vllm

xxll88 opened this issue 11 months ago

NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE

ttkrpink opened this issue 11 months ago

Random VQA Sample button for VLM direct chat

lisadunlap opened this pull request 11 months ago

Update model support

merrymercy opened this pull request 11 months ago

Add psutil to pyproject.toml dependencies

ShukantPal opened this pull request 11 months ago

Failed to import tinychat.

ttkrpink opened this issue 11 months ago

Multi GPU Inference Failure: Rotary Embedding Bug

Tanveer81 opened this issue 11 months ago

GPTQ 4bit Error RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM: 2024-02-12 21:24:19 | ERROR | stderr | Unexpected key(s) in state_dict:

KyrieCui opened this issue 11 months ago

Support Gemini Ultra

youself64github opened this issue 11 months ago

Add llava 34b template

merrymercy opened this pull request 11 months ago

Update version to 0.2.36

merrymercy opened this pull request 11 months ago

Migrate the gradio server to openai v1

merrymercy opened this pull request 11 months ago

ModuleNotFoundError: No module named 'psutil'

ShukantPal opened this issue 11 months ago

Update gradio demo and API model providers

merrymercy opened this pull request 11 months ago

Update gradio version in `pyproject.toml` and fix a bug

merrymercy opened this pull request 11 months ago

Update readme

merrymercy opened this pull request 11 months ago

Upgrade gradio to 4.17

infwinston opened this pull request 11 months ago

Inconsistent use of model_path and model_id in gen_model_answer.py

abgoswam opened this issue 11 months ago

Multi-gpu execution of SGLang

surak opened this issue 11 months ago

python3.8/site-packages/fastchat/serve/base_model_worker.py asyncio error: 'asyncio' has no attribute 'to_thread'

svetaatluxai opened this issue 11 months ago

是否支持npu分布式多卡推理及api部署？

Jakin-huang opened this issue 11 months ago

--state-dict option to overrwide pretrained kwargs

ShukantPal opened this pull request 11 months ago

Update MLX integration to use new generate_step function signature

aliasaria opened this pull request 11 months ago

Jais Support

ShuYuHuang opened this pull request 11 months ago

Update conversation.py to fix “When the type of context in the incoming messages is text, an error”

icowan opened this pull request 11 months ago

Variation in MT-bench score

bidyapati-p opened this issue 11 months ago

When the type of context in the incoming messages is text, an error occurs.

icowan opened this issue 11 months ago

Update model_support.md

infwinston opened this pull request 11 months ago

Trials for supporting AOAI endpoint, Phi2

abgoswam opened this pull request 11 months ago

Update lightllm_integration.md

eltociear opened this pull request 11 months ago

Please help add SeaLLM-7B-v2 with 7.54 on MT-bench on the leaderboard

nxphi47 opened this issue 11 months ago

Support codellama 70B

pydaxing opened this issue 11 months ago

Error while python3 get_model_answer.py

GradientGuru opened this issue 11 months ago

Side-by-side multimodal arena

BabyChouSr opened this pull request 11 months ago

Kind request for updating MT-Bench leaderboards with Qwen1.5-Chat series

Lukeming-tsinghua opened this issue 11 months ago

Fix type hint for play_a_match_single

MonkeyLeeT opened this pull request 11 months ago