github.com/lm-sys/FastChat issues | Ecosyste.ms: OpenCollective

Fix port in worker-address in vllm_worker for #2467

yaqiangsun opened this pull request 8 months ago

stream mode error

day9011 opened this issue 8 months ago

[WIP] Image arena moderation

lisadunlap opened this pull request 8 months ago

Add Multimodal Textbox

BabyChouSr opened this pull request 8 months ago

What is gpt2-chatbot

fakerybakery opened this issue 8 months ago

Leaderboard's Knowledge cutoff format

TechShooter opened this issue 8 months ago

Arena Error - probabilities contain NaN

SuperSecureHuman opened this issue 8 months ago

Error running fastchat.serve.cli ( "safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge" )

Vinooj opened this issue 8 months ago

This problem occurs intermittently, what could be the reason?

cc2017111 opened this issue 8 months ago

[WIP] Fixed Leaderboard dropdown bug and update theme

lisadunlap opened this pull request 8 months ago

[Non-Urgent] Remove Emoji from Leaderboard

HuanzhiMao opened this issue 8 months ago

Support llama3 fine-tune

MrZhengXin opened this pull request 8 months ago

Support Llama-3 fine-tune

MrZhengXin opened this pull request 8 months ago

added mirroring to hf leaderboard

lisadunlap opened this pull request 8 months ago

code update

infwinston opened this pull request 8 months ago

Small fix in clean_chat_data

Ying1123 opened this pull request 8 months ago

Load LoRA for inference

shuishu opened this issue 8 months ago

deepseek-coder-33b-instruct model with openai got "InvalidChunkLength" error

garyyang85 opened this issue 8 months ago

Update gen_model_answer.py

elyhahami18 opened this pull request 8 months ago

why I cant get_conv_template of llama3

AII6 opened this issue 8 months ago

Solve the compatibility problem of SeparatorStyle.CHATML type messes …

icowan opened this pull request 8 months ago

MT-bench (llm_judge) error with Llama3: <class 'openai.error.InvalidRequestError'> This model's maximum context length is 8192 tokens

JianbangZ opened this issue 8 months ago

Solve the compatibility problem of SeparatorStyle.CHATML type messes field

icowan opened this pull request 8 months ago

New questions

rkhazh opened this pull request 8 months ago

Add Japanese for the category

Yuto-24 opened this issue 8 months ago

Question on Continuous Batching Feature in VLLM Worker

un-certainty opened this issue 8 months ago

Fix web launch bug

lvzii opened this pull request 8 months ago

Chatbot Arena: Add phi-2 and phi-3 models

EwoutH opened this issue 8 months ago

Fix ChatGLM3 format error

gc-fu opened this pull request 8 months ago

[Feature Request] Support for Huggingface Chat Templates

PyroGenesis opened this issue 8 months ago

add chat template for CohereForAI/c4ai-command-r-plus

KehaoWu opened this issue 8 months ago

Fix bug in gradio_web_server.py

renning22 opened this pull request 8 months ago

problem about ascend npu 910A

BruceWang1996 opened this issue 8 months ago

gemini-1.5-pro-api-0409-preview model does not support streaming output

Sn0wo2 opened this issue 8 months ago

Finetuning Llama3-8B fails due to lack of a PAD token in new tokenizer

RDouglasSharp opened this issue 8 months ago

SillyTavern ui? FastChat compatible?

QuinnPiers opened this issue 8 months ago

Please support top_logprobs via API

xufengduan opened this issue 8 months ago

Can support the new model about Llama3

1737686924 opened this issue 8 months ago

How deepseekcoder6.7b supports streaming output results

469981325 opened this issue 8 months ago

fastchat模型部署支持qwen1.5吗？

yawzhe opened this issue 8 months ago

dont support CohereForAI/c4ai-command-r-plus-4bit

valueLzy opened this issue 8 months ago

support llama3

gasvn opened this pull request 8 months ago

Change separator in phind conversation template from colon to newline

dannydarvish opened this pull request 8 months ago

adding llama3 template

TJ-Solergibert opened this pull request 8 months ago

[Llama3-Instruct] Conversation template

LeshengJin opened this pull request 8 months ago

Chatbot Arena: Add Reka Edge and Core models

EwoutH opened this issue 8 months ago

GPT-4-turbo MMLU scores?

Duncan-Haywood opened this issue 8 months ago

ValueError: Tokenizer class GemmaTokenizer does not exist or is not currently imported,

1737686924 opened this issue 8 months ago

Very quick question about rule for handling position bias in Agreement statistics

GradientGuru opened this issue 8 months ago

The fix for the Chinese-Alpaca-2-7B model matches the wrong AlpacaAda…

linstreamer opened this pull request 8 months ago

Can Azure openi api be proxied by fastchat.serve.openai_api_server?

Huyueeer opened this issue 8 months ago

[Feature Request] Pre-built docker image support

zsaladin opened this issue 8 months ago

training data?

silvacarl2 opened this issue 8 months ago

Cannot serve `CohereForAI/c4ai-command-r-plus-4bit`

phisad opened this issue 8 months ago

Which version of gpt-4 is used to generate the mt-bench scores on lmsys leaderboard?

YizeMinimax opened this issue 8 months ago

Will the cache kv become invalid?

oslijunw opened this issue 8 months ago

Support NousResearch/Hermes-2-Pro-Mistral-7B

mikutsky opened this issue 8 months ago

Register SmaugChatAdapter.

arkapal3 opened this pull request 8 months ago

Inaccessible Leaderboard with Screenreader

chigkim opened this issue 8 months ago

启动模型的时候指定gpu报错

ilovecomet opened this issue 8 months ago

Azure OpenAI support for MT-Bench

maxim-saplin opened this pull request 8 months ago

Update README.md

infwinston opened this pull request 8 months ago

MMLU for GPT-4-turbo models?

Duncan-Haywood opened this issue 8 months ago

[BUG] RuntimeError: NPU out of memory. Tried to allocate 268.00 MiB

WangxuP opened this issue 8 months ago

Error in sys.excepthook

SAP-BobJi opened this issue 8 months ago

Reka AI model integration

xvblack opened this pull request 8 months ago

Support Gemini Pro 1.5

NightMachinery opened this issue 9 months ago

Question about gen_model_answer.py with gpt2-large

QiyaoWei opened this issue 9 months ago

Analysis

AsaCooperStickland opened this pull request 9 months ago

H2O-Danube template & minor fixes

psinger opened this pull request 9 months ago

How can I use Multiple NPUs ?

QuentinWang1 opened this issue 9 months ago

upgrade xFasterTransformer support to latest version

a3213105 opened this pull request 9 months ago

Gradio UI 0.2.36 not launching

stephanbertl opened this issue 9 months ago

Frontend svelte-i18n error when chatting with custom API models

acylam opened this issue 9 months ago

[Usage] How to inference with multi-GPUs in single machine? Possible to do batch inference?

ee2110 opened this issue 9 months ago

chore: let conv.message be str if no image provided in request

Lanture1064 opened this pull request 9 months ago

How to register request style api to serve on web UI?

congchan opened this issue 9 months ago

GPU memory keeps growing with more talks, but the GPU memory is not released when the tallk is closed.

0Xiao0 opened this issue 9 months ago

Experiment26 Model (7B) Better than Starling Beta? Please Add.

JamesClarke7283 opened this issue 9 months ago

multi-node training not working

ashmalvayani opened this issue 9 months ago

Update README.md

infwinston opened this pull request 9 months ago

[feature] support codellama 70b

kota-iizuka opened this pull request 9 months ago

Add code tagger

Naman-ntc opened this pull request 9 months ago

when finetune 13B model it will have error on safe_save_model_for_hf_trainer

Oscarjia opened this issue 9 months ago

How to set random seed number to get consistent output on Vicuna-13b-v1.3?

ee2110 opened this issue 9 months ago

Print SEED Value for each response [Feature request]

deadpipe opened this issue 9 months ago

Better device_map and max_memory for loading vicuna model

sleeepeer opened this issue 9 months ago

Inconsistency of prompt for llama2

XuHwang opened this issue 9 months ago

JSON decode error

0sengseng0 opened this issue 9 months ago

Add support for Smaug-2.

arkapal3 opened this pull request 9 months ago

An idea for multi-response battles

endolith opened this issue 9 months ago

如何进行多卡多机，有没有详细教程

bravelyi opened this issue 9 months ago

Make Claude 3 haiku as default model, instead sonnet

DarkWyvBat opened this issue 9 months ago

It is possible to run a worker using API-based models

harryzwh opened this issue 9 months ago

is it possible to use the same controller and openai_api_server for multiple model_worker ?

yananchen1989 opened this issue 9 months ago

[embedding model & connect with Langchain] "POST /v1/embeddings HTTP/1.1" 500 Internal Server Error

panxnan opened this issue 9 months ago

Remove Bard disclaimer

abitrolly opened this pull request 9 months ago

Add `--max-model-len` param and description to `serve.vllm_worker`?

Lanture1064 opened this issue 9 months ago

support repetition_penalty param in vllm_worker

cybergeek2077 opened this pull request 9 months ago

Change the controller heart beat thread to daemon

tumbarumba opened this pull request 9 months ago