Bug Report start streaming causes instant crash/coredump

JonnoN

JonnoN

New Member
Streaming from a RTSP IP camera to youtube. This used to work. Possible change is that I'm now scaling the video resolution.
Log, backtrace, and memory map attached. This is obs studio 22.0.3 on Fedora 29 (64-bit, kernel 4.20.3-200)

malloc(): smallbin double linked list corrupted
Aborted (core dumped)
 

Attachments

  • OBS log.txt
    12.3 KB · Views: 76
  • OBS mem map.txt
    153.1 KB · Views: 42
  • OBS backtrace.txt
    153.2 KB · Views: 50
  • OBS console output.txt
    16.8 KB · Views: 55
Tuna

Tuna

Member
Looks like you are using secure rtmp for transmission? The crash happens inside the mbedtls library of your fedora install it seems. Right at the beginning when it does the key exchange things or something.. Can you try broadcasting to a regular rtmp service?
 
JonnoN

JonnoN

New Member
Just streaming to Youtube, don't know how I would've set it to secure.

However, you pointed me in the right direction... I downgraded from mbedtls-2.16.0-1.fc29 to mbedtls-2.13.0-1.fc29 and now it works again! thanks!
 
XertroV

XertroV

New Member
I had this issue two days ago too - version 22.0.3-3 from rpmfusion; downgrading to 22.0.3-2 fixed the issue.

Symptoms sound the same: crashed immediately after clicking "stream", though recording worked fine.

Didn't get much from running OBS from CLI besides this: `malloc(): invalid size (unsorted)`

Unfortunately was under time pressure so didn't investigate more than I needed.
 
M

mcli

New Member
Hi All,

I believe I traced the problem to certificate loading. embedtls is trying to load .trust.crt files and returning an unknown format, then returning a nonzero value from mbedtls_x509_crt_parse_path. The RTMP_TLS_LoadCerts function then treats it as an error, frees the chain and sets the RTMP_TLS_ctx->cacert = NULL. I believe that a nonzero value should just be regarded as a warning.

Please take a look at my commit at: https://github.com/mcli/obs-studio/commit/d565f5e8671c58badab7065b514a6fe53332fa19

For some reason, when I compiled the obs-studio project from source, the program did not crash when streaming, so I could not reproduce the problem. However, when I made a change to the embedtls code to ignore the trust.crt files and return 0 for the mbedtls_x509_crt_parse_path, the obs-studio binary distribution as installed via rpm no longer crashed when streaming. I have suggested changes to the embedtls project, but feel that the obs-studio code should handle cert loading errors properly as well.

If you agree, I can submit an issue and pull request.
 
You must log in or register to reply here.
Top