Large (>~2GB) uploads fail due to timing issues surrounding temp file copies

[AngusThermo-Pyle]

Describe the current behavior

Uploading a large file via web page or API results in multiple 234 errors from ffmpeg such as:
"ffmpeg exited with code 234: Error opening input file /storage/web-videos/private/e25ddfc8-f3aa-41ab-a422-857db1b35807-1080.mp4.\nError opening input files: Invalid argument\n"

Adding a kludge to dist/core/controllers/api/videos/upload.js to create a 10s delay before processing the upload appears to bypass the issue:
line 1:
import { wait } from '@peertube/peertube-core-utils'
~line 39:
await wait(10000)

My conclusion is that we aren't waiting on the file copy from temp files to complete, which works on small files that take little time, but on large files, the copy hasn't completed when the ffmpeg command is executed, so ffmpeg has nothing to work with yet and errors out. A proper solution is needed that actually waits on the completed copy to cover all possible cases and avoid the artificial delay added here as a temporary workaround.

Steps to reproduce

1. Install a stock base config or use an existing instance
2. Upload a large valid file, ~2GB or greater, depending on disk speed
3. Observe error 234 reported in logs by ffmpeg

Describe the expected behavior

Upload should complete and video should be transcoded and processed as usual and as it does for smaller video files.

Additional information

• Version: 6.3.3
• NodeJS version: 23.3.0
• Ffmpeg version: 6.1.1-3ubuntu5

Can provide details on production.yaml, ngnix config or more server logging, but reproduction seems pretty reliable.

[Chocobozzz]

Hi,

Can you enable debug logs in peertube, retry the upload and paste them?

[AngusThermo-Pyle]

err.log

After a channel sync of some 1500 files, four continue to fail transcoding, all >2GB in size. Debug level log of a transcode attempt on one attached. I haven't tried adding the same jinky delay for the transcode yet, but I suspect it will function as the upload one above. I've also tried switching the PeerTube tmp directory to tmpfs in case the increase in copy speed might work around the timing issue, to no avail.

Here's a link to download the file /storage/web-videos/ec4d045c-25f3-4de1-bae9-644683186b2a-2160.mp4 from the debug log run: https://drive.proton.me/urls/CSP5E2HR8G#3pAWmMhoGjtj

[Chocobozzz]

Please disable your transcoding profile and retry