FFmpeg Integer Narrowing Conversion OOB Memory Access in AV1 RTP Packetizer
Impact
FFmpeg before commit b4c199c contains an incorrect integer narrowing conversion in the AV1 RTP packetizer (libavformat/rtpenc_av1.c). The OBU size is cast to long before comparison against the remaining frame size. On targets where long is 32 bits, including 64-bit Windows, sufficiently large OBU size values are sign-flipped by the narrowing cast, producing a negative value that passes the payload size check. This allows an oversized OBU to bypass the safety bound on affected platforms, leading to out-of-bounds memory access when the oversized value is subsequently used as a copy length.
Source excerpt · VulnCheckAffected packages
Affected products
Vendor version evidence. Product records are separate from automatic dependency matching.
FFmpeg · FFmpeg
Other versions: unaffected
| Version / range | Source status |
|---|---|
| 0 to before b4c199c5906ff53368926c2a5839881f41957e7f | affected |
Original source description
FFmpeg before commit b4c199c contains an incorrect integer narrowing conversion in the AV1 RTP packetizer (libavformat/rtpenc_av1.c). The OBU size is cast to long before comparison against the remaining frame size. On targets where long is 32 bits, including 64-bit Windows, sufficiently large OBU size values are sign-flipped by the narrowing cast, producing a negative value that passes the payload size check. This allows an oversized OBU to bypass the safety bound on affected platforms, leading to out-of-bounds memory access when the oversized value is subsequently used as a copy length.