zlib-rs stack overflow during decompression with malicious input

Summary

A denial of service vulnerability was found in zlib-rs, triggered by specially constructed input. This input causes a stack overflow, resulting in the process using zlib-rs to crash. ### Impact Due to the way LLVM handles the zlib-rs codebase, tail calls were not guaranteed. This caused certain input patterns to result in a large number of stack frames being required, quickly resulting in a stack overflow. These are unlikely to occur in practice, but a dedicated attacker can construct malicious input files. After stack overflows were found by @inahga with a fuzzer, we dove into the assembly, and found some cases where the stack grew ```asm .LBB109_326: mov rdi, rbx call zlib_rs::inflate::State::type_do jmp .LBB109_311 .LBB109_311: lea rsp, [rbp - 40] pop rbx pop r12 pop r13 pop r14 pop r15 pop rbp .cfi_def_cfa rsp, 8 ret ``` LLVM wants to centralize the cleanup before the return (many other blocks jump to `LBB109_311`), thereby invalidating a tail call to `type_do`. We were not able to get rid of this call without introducing one elsewhere: we just don't currently have the power to tell LLVM what we want it to do. So, we switch back to loop+match waiting for changes to rust to make a more efficient implementation possible. Performance-wise, the damage is relatively minimal: we're just slower in cases where we already were slower than C. We are faster in cases where the relevant code is barely touched (in these cases the logic quickly moves into a hot inner loop and just spends most of its time there). ### Patches Version 0.4.0 patches the problem and is no longer vulnerable. ### Workarounds Users of zlib-rs should upgrade to the latest version. Users could alternatively run zlib-rs in a separate process to prevent a stack overflow crashing the entire program. In some situations a signal handler can be used to catch a stack overflow happening.