Large file upload causes high memory usage and container crash

[pscriptos]

Describe the bug
When uploading a large video file (e.g., a .mkv file >7GB) via the web interface, ConvertX appears to load the entire file into RAM during upload. Once the available system memory is exhausted, the container crashes and the web interface becomes unreachable until the container is restarted.

To Reproduce
Steps to reproduce the behavior:

1. Open the ConvertX web interface (I’m accessing it via HTTP on my local Docker host).
2. Click on "Upload" and select a large video file (tested with a 7.3 GB .mkv).
3. Monitor system memory usage (e.g., via htop or docker stats) during the upload process.
4. Once RAM is fully consumed, the container crashes and stops responding.

techniverset_HvWqRXglxJ.mp4

Expected behavior
Files should ideally be streamed to disk rather than loaded entirely into memory. This behavior causes a crash on large file uploads and makes the software unusable for larger videos.

Additional context

• I’m using ConvertX via HTTP on a local Docker host.
• The bug can be reliably reproduced with any large file.
• This issue could potentially be mitigated by changing the upload mechanism (e.g., streaming to temporary file instead of RAM).

Edit:

Additional observations
It appears that not only the upload, but also the conversion process loads the entire file into RAM.
I was able to reproduce the issue with a 900 MB file: during conversion, memory usage increased rapidly, and ConvertX eventually became unresponsive when RAM was exhausted.

This makes ConvertX unusable on systems with limited memory, even for moderately sized files.

techniverse_MobaXterm_Pro_port_25.0_xTmp2dY8WX.mp4

You can scroll through the video to speed things up – it's nearly 8 minutes long, and unfortunately, I wasn’t able to create a proper timelapse.

I've also added a screenshot from my monitoring setup which clearly shows the issue as well.

Docker-Logs

jobId set to: 225
jobId set to: 226
jobId set to: 227
jobId set to: 228
jobId set to: 229
jobId set to: 230
jobId set to: 231
jobId set to: 232
jobId set to: 233
jobId set to: 234
jobId set to: 235
jobId set to: 236
jobId set to: 237
jobId set to: 238
jobId set to: 239
jobId set to: 240
jobId set to: 241
stderr: Killed
magick: no images for write '-write' './data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4' at CLI arg 2 @ error/operation.c/CLINoImageOperator/4985.

Failed to convert ./data/uploads/96165594783682/200/Balduin, der Geldschrankknacker (1964).avi from avi to mp4 using imagemagick. error: Error: Command failed: magick ./data/uploads/96165594783682/200/Balduin, der Geldschrankknacker (1964).avi ./data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4
Killed
magick: no images for write '-write' './data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4' at CLI arg 2 @ error/operation.c/CLINoImageOperator/4985.

jobId set to: 242
jobId set to: 243
jobId set to: 244
jobId set to: 245
ENOENT: no such file or directory, open './data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4'
    path: "./data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4",
 syscall: "open",
   errno: -2,
    code: "ENOENT"

jobId set to: 246
jobId set to: 247
jobId set to: 248
jobId set to: 249
jobId set to: 250
jobId set to: 251
ENOENT: no such file or directory, open './data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4'
    path: "./data/output/96165594783682/200/Balduin, der Geldschrankknacker (1964).mp4",
 syscall: "open",
   errno: -2,
    code: "ENOENT"

Thank you for taking a look at it.
If you need anything else from me, please let me know.

Otherwise, I have to say that this is a really great project!
Thank you for it!