401 Commits

Author SHA1 Message Date
dependabot[bot]
d6827b5aa2
Update goblin requirement from 0.7 to 0.8 (#2232)
Updates the requirements on [goblin](https://github.com/m4b/goblin) to permit the latest version.
- [Changelog](https://github.com/m4b/goblin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/m4b/goblin/commits)

---
updated-dependencies:
- dependency-name: goblin
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 14:21:25 +02:00
Dominik Maier
117741e1b1
Run Clippy on macOS (#2222)
* macos clippy

* Fix clippy on macOS
2024-05-20 02:58:42 +02:00
Romain Malmain
dfd3b3278e
QEMU command system refactoring (#2189)
* implemented generic command builder

* Added builder to `Emulator`.
2024-05-18 20:43:56 +02:00
dependabot[bot]
8d54ee2c80
Update pyo3-build-config requirement from 0.18 to 0.21 (#2205)
Updates the requirements on [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.18.0...v0.21.2)

---
updated-dependencies:
- dependency-name: pyo3-build-config
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-05-18 20:22:35 +02:00
dependabot[bot]
7cf821bd82
Update toml requirement from 0.4.2 to 0.8.13 (#2195)
---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-16 14:06:53 +02:00
dependabot[bot]
e70b23fc6b
Update serde_yaml requirement from 0.8 to 0.9 (#2185)
Updates the requirements on [serde_yaml](https://github.com/dtolnay/serde-yaml) to permit the latest version.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.0...0.9.34)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-15 15:17:38 +02:00
Romain Malmain
b127f0579d
Regenerate QEMU binding stubs only for newer versions of the nightly compiler (#2177)
* Regenerate binding stubs only for newer versions of the nightly compiler.

* fmt

* clippy
2024-05-15 01:42:59 +02:00
dependabot[bot]
dce0761b11
Update capstone requirement from 0.11.0 to 0.12.0 (#2170)
Updates the requirements on [capstone](https://github.com/capstone-rust/capstone-rs) to permit the latest version.
- [Release notes](https://github.com/capstone-rust/capstone-rs/releases)
- [Changelog](https://github.com/capstone-rust/capstone-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/capstone-rust/capstone-rs/compare/capstone-v0.11.0...capstone-v0.12.0)

---
updated-dependencies:
- dependency-name: capstone
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-05-13 21:09:02 +02:00
Romain Malmain
b0d95676f9
Fix regex fixing generated bindings for QEMU (#2163)
* Fix regex fixing generated bindings

* Do not write 2 times bindings to filesystem

* Update stub bindings

* fmt

* clippy

* fmt

* use `unsafe extern "C"` instead of `extern "C"`.
2024-05-13 17:41:56 +02:00
Romain Malmain
edb6b509c2
Generate stub bindings less aggressively. (#2164)
Emit warning when using runtime files for an incompatible architecture.
2024-05-13 11:58:19 +02:00
Kevin Phoenix
3328f06de4
Add qemu runtime defs for aarch64 (#2151) 2024-05-12 15:39:55 +02:00
Addison Crump
614b149cae
Fix cfg uses across codebase (#2154)
* start fixup of cfgs

* whoops

* remaining issues

* maybe fixup macos

* apparently that is ios code?

* fix no_std

* Re-enable tcp_compression

* fix tcp broker

* remove tcp_debug flag

* fmt

* clippy

* less unwrap while we're at it

* doc

* add back pub star uses for libafl_qemu

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-05-08 14:31:10 +02:00
Dominik Maier
c97c8cf83e
Rename Handler -> Handled, various other cleanups after #2142 (#2152)
* Handle name cleanup after #2142

* Handleable -> CanBeHandled

* CanBeHandled -> Handled

* clippy

* forgot one

* Handled more
2024-05-07 19:23:13 +02:00
Romain Malmain
bed500471a
QEMU generic memory iterator (#2148)
* QEMU generic memory iterator + Refactoring

* Generic Memory Iterator (systemmode only for now): It is now possible to iterator over memory ranges, independently of the address kind

* Refactoring or Emulator / Qemu structures: they are now handled separately in different files

* Refactoring of Exit Handlers: Result / Error structs have been clarified

* Simple handler for signals

* add new `check-cfg` calls for libafl qemu
2024-05-07 10:46:15 +02:00
cube0x8
a16fb88f3e
Add null ptr to devices vec (QEMU systemmode) (#2141) 2024-05-06 23:19:30 +02:00
Dongjia "toka" Zhang
4562f19bdc
Qemu ctrl c exit (#2147)
* fix

* ignore

* fix

* aaa

* bbb
2024-05-06 15:42:43 +02:00
Dongjia "toka" Zhang
b0ac25a7b5
Rename Reference to Handle (#2142)
* rename

* rename

* fmt

* fix

* fmt
2024-05-04 18:16:28 +02:00
Romain Malmain
2302f1b35c
Fix QEMU doc not building (again) (#2130)
* fix doc not building.
2024-05-02 15:17:35 +02:00
cube0x8
c79cdc89ac
add mmap hooks to libqasan (#2122)
* add mmap hooks to libqasan

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-05-02 10:39:33 +02:00
Romain Malmain
935100e7a7
Qemu update to latest version + various fixes (#2119)
* Fix maps iterator

* Added paranoid debug feature.

* Fix snapshot bugs.

* Update qemu-libafl-bridge.

* Added auto-generation of `x86_64_stub_bindings.rs` .

* Fix sync_exit calling to stopped CPU.
2024-05-02 09:31:54 +02:00
Dongjia "toka" Zhang
50f14d44fa
Restrict more on edges map API (#2117)
* fix

* fix

* feature guard

* use

* cannnot find

* fmt

* more

* fix

* fix

* more

* fucking fmt just shut

* rename and change stuff

* fix

* fix

* fixxxxxxxxxx
2024-04-29 16:28:40 +02:00
Addison Crump
28c43b332f
Match by Ref fix (#2105)
* match by ref fix

* impl Named for Reference

* rename

* magic indexing

* whoops

* docs, clippy

* some additional CI complaints

* other libafl_qemu fixes

* missed an alloc feature

* a smattering of fixes

* use from not direct construction

* tinyinst fix

* horrible double-mutability things

* fixup nyx

* from not new

* forkserver_simple fixes

* dogfood: forkserver

* mmmm yummy dogfood

* round one CI fixes

* clippy appeasement

* deref generic impl to simplify usage

* adaptive serialization (ouch)

* remaining clippy items

* I am tired

* new not with

* fixup: aflpp tracing was not actually constructable

* fix tmin

* reduce complexity of map feedback now that we do not need to constrain

* frida fixes

* fix concolic

* type_ref => reference
2024-04-27 18:08:09 +02:00
Romain Malmain
dd0fbff819
Small refactoring of libafl_qemu structure (#2098)
* Architecture-specific stuff is now in the `arch` directory

* Helpers are now in the `helpers` directory

* `emu.rs` has been moved as `emu/mod.rs` for consistency with the rest of the repository
2024-04-24 21:47:29 +02:00
Stefan Zabka
0f42efa12b
refactor(snapshot.rs): use const generics (#2069)
* refactor(snapshot.rs): use const generics

* Add docs to read hooks

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-04-24 11:58:46 +02:00
Dongjia "toka" Zhang
176659821a
Add TypeRefCreator trait (#2065)
* poc

* ai suggestion

* rename this

* aaaa

* fmt

* simplify

* delete blob

* ignore

* fixup?

* some progress on cow-ification

* some more

* clippy fixes, finalise tests

* whoops, missed a spot

* no std compat

* api change: Named now requires alloc feature

* doc fix

* missed a spot

* additional fixes

* libfuzzer fixes

* fix tutorial

* fix

* add

* aa

* fix tutorial

* fix

* Rename

* fix

* aa

* fmt

* aa

---------

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2024-04-23 19:03:30 +02:00
Romain Malmain
fef6c8d1b1
Run qemu_systemmode tests with self-hosted runners (#2018)
* run qemu fuzzers (qemu_systemmode only for now) in self-hosted runners

* Remove qemu-related fuzzers to general fuzzers

* fix

* Install dependencies before anything else

* Do not use sudo

* Install sudo

* Revert "Install dependencies before anything else"

This reverts commit 107addad5d9f68dec5a9af50831112cd72c28f4d.

* added qemu specific prerequisites

* add -y flag

* Format with nightly

* Do not use nightly only.
Install fmt and clippy for stable as well.

* Install qemu-img for qemu

* fix qemu-img install

* apt update

* Changed timeout.

* Fix qemu_systemmode test

* fmt

* clippy + decorrelate build and run for qemu_systemmode.

* fix fuzzer

* clippy

* add sqlite3-dev to package prerequisites.

* add arm-none-eabi-gcc

* fix profile dir

* fix condition.

* Run less QEMU stuff faster

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-04-22 18:17:22 +02:00
Dongjia "toka" Zhang
e8cc1ba93d
Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use" (#2082)
* Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2…"

This reverts commit c68b3816fb680b635f99c337f78185e699864705.

* all

* more
2024-04-19 19:03:58 +02:00
Dongjia "toka" Zhang
c68b3816fb
Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2079)
* fix

* aa

* push

* fix

* fmt
2024-04-19 13:24:25 +02:00
Dongjia "toka" Zhang
c50af44099
Split edges_map_size and edges_map_size_alloc (#2012)
* two sizes

* rename

* build.rs

* rename

* capital

* FMT

* macro

* fuck

* fff
2024-04-17 17:58:08 +02:00
Romain Malmain
bc3ef5952b
Rework of libafl_qemu configuration (#2054)
* LibAFL QEMU can now be dynamically linked
* LibAFL QEMU reconfiguration happens less frequently (now using a signature check)
* Possibility to have custom rpath in QEMU
2024-04-16 11:35:15 +02:00
Dongjia "toka" Zhang
cbb323f26c
LibAFL 0.12 (#1823) 2024-04-12 18:42:14 +02:00
Romain Malmain
ec935bf95f
Fix GuestMaps iterator in LibAFL QEMU. (#2041)
* Fix maps iterator.

* Use native QEMU structs instead of pointer casting.

* Update stub bindings.

* Maps operations stored in usermode.
2024-04-12 14:40:53 +02:00
Romain Malmain
94a2a2363a
Add CPUArchStatePtr to backdoor signature (#2038) 2024-04-11 17:30:59 +02:00
Stefan Zabka
48463d079b
Fix error handling in libafl_qemu_build (#2036)
* fix(libafl_qemu_build): assert command success

* fix(libafl_qemu_build): make sure linker_interceptor.py picks up correct compiler

Currently linker_interceptor.py uses 'cc' as the __LIBAFL_QEMU_BUILD_CC environment variable is never set

* remove redudant arg
2024-04-10 18:36:36 +02:00
WorksButNotTested
374f8735fa
Implement user-space QEMU ASAN (#1806)
* Implement user-space QEMU ASAN

* Fix wrong cfgs

* fmt

* merge conflicts in libafl qemu

* A few more fixes to qemu_launcher

* Change commit of qemu-libafl-bridge

* Fix clippy in qemu_launcher

* Fix commit id again

* Empty commit to trigger CI

* Fix path to fuzzer for test in qemu_launcher?

* Revert location of target binary and show the full error log from qemu_launcher test

* Appease the clippy gods

* Empty

* Fix format

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2024-04-10 01:11:28 +02:00
Romain Malmain
33c6ea1ae4
Fix libafl_qemu python bindings and example (#2030) 2024-04-09 16:09:47 +02:00
Mrmaxmeier
f252469d86
CI: A few updates and fixes (#2023)
* CI: Don't run cargo clean in order to preserve build caches

* CI: run yaml formatter >:)

* CI: try sharing build caches between similar jobs

* CI: it looks like the `ucd-generate` tool is not used anymore

* CI: Sneak in merge queue support :)

* CI: split out `cargo fmt --check` job

* libafl_qemu injections: be more resilient about mapping paths
2024-04-09 12:55:07 +02:00
Dominik Maier
7479726c3e
Un-Unfix libafl_qemu fixes after #2020, remove python CI (#2024)
* Un-Unfix libafl_qemu fixes after #2020

* remove CI for removed python bindings
2024-04-08 22:06:40 +02:00
Dominik Maier
f19302c9b1
Bring back python bindings for sugar,qemu (partially revert #2005) (#2020)
* Bring back python bindings for sugar,qemu (partially revert #2005)

* sugarman, won't you hurry

* Test?
2024-04-08 19:36:54 +02:00
Dongjia "toka" Zhang
e8fe5bb614
Move HasMetadata/HasNamedMetadata trait to libafl_bolts (#2016)
* move around

* fmt

* rev

* fmt

* doc job

* name

* move to libafl

* separate even more

* miri
2024-04-08 19:24:56 +02:00
Dominik Maier
ff938261df
Fix Clippy on MacOS for qemu, accept some clippy::missing_transmute_annotations suggestions (#2009)
* Fix Clippy/CI

* Ignore transmute annotations in pybind

* fix

* more clippy

* more clippy

* fix fixes
2024-04-08 16:24:10 +02:00
Andrea Fioraldi
e29897dd44
Fix GuestMaps in userspace LibAFL QEMU (#2013) 2024-04-08 08:30:23 +02:00
Dongjia "toka" Zhang
527b892c1d
Separate fuzzer test in CI (#2010)
* ci

* ci

* ci

* aa

* aa

* a

* a

* a

* fix

* no fail fast

* system

* what is this lint...

* a

* clp

* disk

* ok no fastfail

* no qemu for now, reenable monday
2024-04-06 21:01:57 +02:00
Dongjia "toka" Zhang
4901846588
Remove python bindings (#2005)
* delete python bindings

* fix

* example

* fix
2024-04-06 15:16:46 +02:00
Romain Malmain
50be2f4aaf
Fix of libafl qemu runtime files (#1996)
* Change naming and fix LibAFLQEMU header file. Fix and document LibAFL QEMU Windows ASM file.

* fmt
2024-04-03 11:57:48 +02:00
Romain Malmain
cca4e3daa9
Make inputs in libafl_qemu generic (#1991)
* fix non-generic input in libafl_qemu

* Update setup_libxml2.sh of nyx_libxml2_standalone fuzzer (#1990)

On testing this on my machine not all neccessary files for fuzzing with nyx where provided by the setup. Compared to the nyx_libxml2_parallel fuzzer this packer generation line was missing.
After adding this the fuzzer was able to start.

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Tobias Mayer <arbeitskraftmayer@gmail.com>
2024-04-02 12:30:12 +02:00
Romain Malmain
50e0f80d1e
Refactoring of qemu_systemmode fuzzers (#1980)
* Added backdoors to portable header file.

* fix arm macros.

* copy `libafl_qemu.h` in target directory.

* Merged all qemu_systemmode examples into one, settable with features.
Automatic building using `Makefile.toml`

* fix typo.

* added test of QEMU systemmode fuzzers.

* replace test by build for now.
2024-03-30 17:04:37 +01:00
Romain Malmain
44c841ffb1
WIP: QEMU exit handler (#1745)
* Added paging filtering.
Reworked address range filtering to fit with new generic code.

* Fix: renamed remaining QemuInstrumentationFilter instances.

* Renamed sync breakpoint to sync exit.

* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.

* sync_backdoor.rs removal.
Formatting.

* Updated `bindgen` and `which`.
Adapting code to work with update.

* fix: reconfigure cleanly if prior configure was interrupted abruptly.

* Enable sanitizers in QEMU during debug.

* Added target-usable files.

* Added breakpoint structure.

* Adapted other files to work with ExitHandler.

* Adapted existing fuzzer to work with new exit handler.

* fix: use get to avoid crashes.

* Updated README to indicate cargo-make should be installed.

* Added QEMU internal exit handler.

* Adapted qemu_systemmode example with new exit handler.

* Fixed fuzzers to work with new exit handler.

* Trying to fix CI (#1739)

* test

* dummy

* dummy

* Added new examples.

* Forgot to add build scripts.

* format

* format

* clang-format

* python emulator adaptation.

* fixed python bindings.

* clippy fixes.

* python bindings.

* fix qemu_sugar.

* fix fuzzbench.

* fixed import issues.

* misc fixes.

* renamed crate.

* Updated x86_64 stub bindings.

* Fixed static naming.

* binding fmt

* clippy

* clippy

* Removed useless return statement.

* removed advice to install cargo-make in individual repositories.

* symcc_update (#1749)

* Remove unused create_anymap_for_trait macro (fixes #1719) (#1752)

* Fix `as_object` UB discussed in #1748 (#1751)

* Fix as_object UB discussed in #1748

* More cleanup, more less UB

* Fix fixes

* Added uninit_on_shmem api

* clippy

* fmt

* trying to fix fuzzers, libfuzzer wrapper

* Add OwnedRefMit::owned constructor, libfuzzer fix

* Some more fixes

* Add BacktaceObserver::owned fn

* fmt

* more fmt

* Ignore SigPipe by default (#1741)

* Ignore SigPipe by default

* Fix no_std

* fmt

* Fix incorrect imports (#1758)

* Fix incorrect imports

https://doc.rust-lang.org/core/simd/trait.SimdOrd.html

* Fix

* Try fix ci

* Documentation fixes (#1761)

* Documentation fixes

* Fix InProcessExecutor url

* Update all urls to latest

* Miri ignores for M1 regex (#1762)

* Enabling DrCov on Windows (#1765)

* Enabling DrCov for Windows

* Dedup common code in scheduler (#1702)

* dedup common code in scheduler

* del eco

* fixing

* fix

* replace `Emulator::new_empty` by `Emulator::get` calls outside of `emu.rs` for safety. (#1763)

* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, and more (#1754)

* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, set_error_print_panic_hook

* Trying to fix #1753

* typo

* More fix

* Fix test?

* more testcase fixes

* Fix: renamed remaining QemuInstrumentationFilter instances.

* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.

* format

* format

* format

* Replace sync_exit with sync_backdoor.

* Rework command system.

* fix bad import.

* format.

* cargo fmt

* disable af-xdp as well to avoid linking errors.

* End of merging.

* format.

* Adaptation for usermode.

* format.

* injection support.

* usermode fixes.
format.

* clippy

* clippy + format

* Do not unwrap emu + format.

* fix: entry_point breakpoint

* inital commit.

* clippy

* tests

* clippy

* adapt example

* systemmode.

* renaming

* fmt

* fix lints.

* more lint fix.

* even more lint fixes.

* always more lint fixes.

* lint fix.

* allow unused qualifications for crate when it could be confusing.

* Still lint fixes.

* Lint fixes on generated code.

* Some lint fixes.

* merge continue.

* renamed modules as well.

* fixing merge.

* systemmode compiling.

* fmt

* fix early emulator drop.

* fmt

* fix cast to c_void of the wrong object.

* Added global enum for snapshot managers.
Some renaming.

* move things around.

* WIP: generic inclusion of QEMU Executor in exit handler.

* * Moved extern calls to `libafl_qemu_sys`
* Replaced old `Emulator` by `Qemu` and only kept C functions wrappers
* Now `Emulator` is for higher-level interactions with QEMU. Kept old functions for compatibility calling to `Qemu` functions
* A direct side effect of this slit is the removal of the `IsEmuExitHandler` trait dependency added in many parts of the code.
* Removed old dirty casting for `QemuExecutor` helpers and used the brand-new access to `QemuExecutorState` instead.
* Minor changes to `Qemu` and `Emulator` `get` methods for cleaner getters.

* Add missing `Qemu` function.

* Updated `qemu_systemmode` example.

* Adapted QEMU fuzzers + renaming.

* Fixed python.

* fix libafl_sugar with new implementation.

* fix dangling RefCell.
adapt new examples.
TODO: merge `libafl_systemmode.*` examples.

* clippy.

* fix more fuzzers.

* clippy.

* Implement `HasInstrumentationFilter` generically.
Updated `StdInstrumentationFilter` accordingly.

* Renamed breakpoint functions for QEMU.
`qemu.run()` handling.

* Removed OnceCell / RefCell in signature.
more explicit `MmapPerms` method names.

* minor code refactoring

* Emulator::run_handle refactoring

* deprecated Emulator functions calling directly to QEMU functions.

* IsSnapshotManager -> SnapshotManager

* IsEmuExitHandler -> EmuExitHandler + fmt

* Generic register when it makes sense.

* reverted IsSnapshotManager -> SnapshotManager because of a collision.

* fix syntax + clippy

* fmt

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: lazymio <mio@lazym.io>
Co-authored-by: Bet4 <0xbet4@gmail.com>
Co-authored-by: mkravchik <mkravchik@hotmail.com>
2024-03-22 18:03:29 +01:00
Dominik Maier
a0143d02a6
Rename OSError -> OsError and merge with redundant Error::File (#1944)
* OSError -> OsError

* Move File errors to io Errors

* Removing File errors

* fixes :)

* More format

* fix libfuzzer runtime
2024-03-18 08:46:48 +01:00
Stefan Zabka
fbd0d497d4
Refactor(build.rs): extract common code between branches (#1950) 2024-03-17 21:31:30 +01:00