Gallopsled / pwntools / 7846247323 / 1

74%

dev: 74%

push

github

Fix Unicorn Engine 1GB limit that calls exit: raise OSError instead (Fixes #2343) (#2347)

* Fix Unicorn Engine 1GB limit exit(): raise OSError instead (Fixes #2343)

This commit fixes #2343: an issue where `pwn checksec <binary>` would fail with a bogus error of:

```
$ pwn checksec /root/x/bin/main
Could not allocate dynamic translator buffer
```

This actually comes from Unicorn Engine which tries to allocate a 1GB RWX mapping:
```
root@pwndbg:~# strace -e openat,mmap pwn checksec /root/x/bin/main 2>&1 | tail -n 10
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/mpmath-0.0.0.egg-info/PKG-INFO", O_RDONLY|O_CLOEXEC) = 7
openat(AT_FDCWD, "/usr/local/lib/python3.10/dist-packages/unicorn/lib/libunicorn.so.2", O_RDONLY|O_CLOEXEC) = 7
mmap(NULL, 22447520, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f2604f9d000
mmap(0x7f2605339000, 13496320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x39c000) = 0x7f2605339000
mmap(0x7f2606018000, 3039232, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x107b000) = 0x7f2606018000
mmap(0x7f26062fe000, 1601536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x1360000) = 0x7f26062fe000
mmap(0x7f2606485000, 525728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2606485000
mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
Could not allocate dynamic translator buffer
+++ exited with 1 +++
```

...and if it fails, it calls `exit()`. This can be seen in Unicorn Engine code: https://github.com/unicorn-engine/unicorn/blob/56f3bdedb/qemu/accel/tcg/translate-all.c#L960-L963

This issue has been reported to Unicorn Engine in https://github.com/unicorn-engine/unicorn/issues/1766 but since it hasn't been fixed, this commit applies a workaround for it.

* CI: add test for pwn checksec with 500MB limit for UE 1GB limit

* Add changelog en... (continued)

4599 of 7354 branches covered (0.0%)

12910 of 17452 relevant lines covered (73.97%)

0.74 hits per line