[BIGSLEEP-434615384] cups-filters 1.x: out of bounds write in pdftoraster

Summary

cups-filters contains backends, filters, and other software required to get the cups printing service working on operating systems other than macos. In cups-filters prior to 1.28.18, by crafting a PDF file with a large `MediaBox` value, an attacker can cause CUPS-Filter 1.x’s `pdftoraster` tool to write beyond the bounds of an array. First, a PDF with a large `MediaBox` width value causes `header.cupsWidth` to become large. Next, the calculation of `bytesPerLine = (header.cupsBitsPerPixel * header.cupsWidth + 7) / 8` overflows, resulting in a small value. Then, `lineBuf` is allocated with the small `bytesPerLine` size. Finally, `convertLineChunked` calls `writePixel8`, which attempts to write to `lineBuf` outside of its buffer size (out of bounds write). In libcupsfilters, the maintainers found the same `bytesPerLine` multiplication without overflow check, but the provided test case does not cause an overflow there, because the values are different. Commit 50d94ca0f2fa6177613c97c59791bde568631865 contains a patch, which is incorporated into cups-filters version 1.28.18.