[Bug] vercel 部署后，无法使用流式传输。

[sheng-di]

Describe the bug
vercel 部署后，无法使用流式传输。
同样的 api base 和 key，直接在设置界面填写，通过浏览器端发起是可以流式传输的。
通过 vercel 部署后，貌似是后端发起的，无法流式传输。

To Reproduce
vercel 部署即可复现。采用环境变量内置的 api_base 和 key。

Expected behavior
能够流式传输。

Screenshots
If applicable, add screenshots to help explain your problem.

Deployment

• Docker
• Vercel
• Server

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional Logs
Add any logs about the problem here.

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Title: [Bug] After vercel is deployed, streaming cannot be used.

[sheng-di]

经过排查，发现是使用的 IIS 的反向代理不支持流式传输。。。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

After investigation, it was found that the reverse proxy of IIS used does not support streaming. . .

[sheng-di]

Update:
修改 api base，修改为直连，不经过反向代理后，仍然不支持流式传输。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Update:
After modifying the api base to direct connection without going through a reverse proxy, streaming is still not supported.

[fredliang44]

This may be due to the recent instability of the Google Gemini Pro model, so you can wait until the evening in APAC to try to confirm the issue. I can't reproduce it on my side at the moment, and the latest commit deployed with Vercel still streams the output normally

[sheng-di]

This may be due to the recent instability of the Google Gemini Pro model, so you can wait until the evening in APAC to try to confirm the issue. I can't reproduce it on my side at the moment, and the latest commit deployed with Vercel still streams the output normally

No, it's not Gemini, it's the back end of chatgpt. The nextchat client of this machine requests the back end of the server, which supports streaming. The nextchat deployed in vercel requests the same back end and does not support streaming. I suspect it is a proxy forwarding problem in the back-end part of nextjs. BTW, the backend is deployed using copilot-gpt4-service.

[sheng-di]

This may be due to the recent instability of the Google Gemini Pro model, so you can wait until the evening in APAC to try to confirm the issue. I can't reproduce it on my side at the moment, and the latest commit deployed with Vercel still streams the output normally

I think it's related to vercel/next.js#9965

[sheng-di]

In app/api/common.ts:

// The latest version of the OpenAI API forced the content-encoding to be "br" in json response
    // So if the streaming is disabled, we need to remove the content-encoding header
    // Because Vercel uses gzip to compress the response, if we don't remove the content-encoding header
    // The browser will try to decode the response with brotli and fail
    newHeaders.delete("content-encoding");

But it doesn't seem to be work.