Skip to content

[Bug]: 使用第三方汇聚API时,出现不兼容Claude的情况。 #1176

@crosswk

Description

@crosswk

这个bug是否已存在现有issue了?

  • 我确认没有已有issue,且已阅读常见问题

错误表现

使用GPT模型没有问题,使用Claude相关模型会有报错。
已经使用12.04号更新的版本依然会有报错。

复现操作

实现了两个厂商的汇聚API,对于Claude模型都有报错。
测试Chat-Next和Lobe没有报错。
麻烦有空可以看下。

我设置的两个厂商是:"openai_api_base": "https://api.xiaoai.plus/v1",和"openai_api_base": "https://s.lconai.com/v1",

错误日志

2024-12-05 01:28:46,180 [INFO] [base_model.py:615] 用户的输入为:output"test"
Traceback (most recent call last):
  File "/chu/Dev-ChuanhuChatGPT/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 244, in _decode_chat_response
    if chunk_length > 6 and "delta" in chunk["choices"][0]:
IndexError: list index out of range
ERROR: {'id': '', 'object': 'chat.completion.chunk', 'created': 1733362128, 'model': 'claude-3-5-sonnet-20240620', 'system_fingerprint': '', 'choices': [], 'usage': {'prompt_tokens': 21, 'completion_tokens': 0, 'total_tokens': 21, 'prompt_tokens_details': {'cached_tokens': 0, 'text_tokens': 0, 'audio_tokens': 0, 'image_tokens': 0}, 'completion_tokens_details': {'text_tokens': 0, 'audio_tokens': 0}}}
Traceback (most recent call last):
  File "/chu/Dev-ChuanhuChatGPT/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 244, in _decode_chat_response
    if chunk_length > 6 and "delta" in chunk["choices"][0]:
IndexError: list index out of range
ERROR: {'id': '', 'object': 'chat.completion.chunk', 'created': 1733362128, 'model': 'claude-3-5-sonnet-20240620', 'system_fingerprint': '', 'choices': [], 'usage': {'prompt_tokens': 21, 'completion_tokens': 0, 'total_tokens': 21, 'prompt_tokens_details': {'cached_tokens': 0, 'text_tokens': 0, 'audio_tokens': 0, 'image_tokens': 0}, 'completion_tokens_details': {'text_tokens': 0, 'audio_tokens': 0}}}
JSON解析错误,收到的内容: data: [DONE]
2024-12-05 01:28:50,696 [INFO] [base_model.py:703] 回答为:
2024-12-05 01:28:50,696 [INFO] [base_model.py:709] Tokens per second:4.882235892234999

运行环境

  • OS:
  • Browser:
  • Gradio version:
  • Python version:

帮助解决

  • 我愿意协助解决!

补充说明

image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions