507 Commits

Author SHA1 Message Date
Valentin Huber
d8ec991b48
Add bloom filter for duplicate execution of the same inputs (#2771)
* fixing empty multipart name

* fixing clippy

* New rules for the contributing (#2752)

* Rules

* more

* aa

* Improve Flexibility of DumpToDiskStage (#2753)

* fixing empty multipart name

* fixing clippy

* improve flexibility of DumpToDiskStage

* adding note to MIGRATION.md

* Update bindgen requirement from 0.70.1 to 0.71.1 (#2756)

Updates the requirements on [bindgen](https://github.com/rust-lang/rust-bindgen) to permit the latest version.
- [Release notes](https://github.com/rust-lang/rust-bindgen/releases)
- [Changelog](https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/rust-bindgen/compare/v0.70.1...v0.71.1)

---
updated-dependencies:
- dependency-name: bindgen
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* No Use* from stages (#2745)

* no from stage

* fixer

* doc fix

* how was this working????

* more fixes

* delete more

* rq

* cargo-fuzz

* m

* aa

* Update CONTRIBUTING.md MIGRATION.md (#2762)

* No Uses* from `fuzzer` (#2761)

* go

* fixing stuf

* hello from windows

* more

* lolg

* lolf

* fix

* a

---------

Co-authored-by: Your Name <you@example.com>

* Remove useless cfgs (#2764)

* Link libresolv on all Apple OSs (#2767)

* Somewhat ugly CI fix... (#2768)

* Maybe fix CI

* does this help?

* Very dirty 'fix'

* Add Input Types and Mutators for Numeric Types (#2760)

* fixing empty multipart name

* fixing clippy

* New rules for the contributing (#2752)

* Rules

* more

* aa

* Improve Flexibility of DumpToDiskStage (#2753)

* fixing empty multipart name

* fixing clippy

* improve flexibility of DumpToDiskStage

* adding note to MIGRATION.md

* Introduce WrappingMutator

* introducing mutators for int types

* fixing no_std

* random fixes

* Add hash derivation for WrappingInput

* Revert fixes that broke things

* Derive Default on WrappingInput

* Add unit tests

* Fixes according to code review

* introduce mappable ValueInputs

* remove unnecessary comments

* Elide more lifetimes

* remove dead code

* simplify hashing

* improve docs

* improve randomization

* rename method to align with standard library

* add typedefs for int types for ValueMutRefInput

* rename test

* add safety notice to trait function

* improve randomize performance for i128/u128

* rename macro

* improve comment

* actually check return values in test

* make 128 bit int randomize even more efficient

* shifting signed values

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>

* Add HashMutator

* Fix docs

* Fix docs again

* introducing bloom filter

* fix tests

* Implement evaluate_filtered

* Add macros to libafl_bolts tuples for mapping and merging types (#2788)

* Add macros

* Use the macros for havoc_mutations

* Fix docs

* improve merge_tuple_list_type to accept n items

* libafl_cc: Automatically find llvm_ar path (#2790)

* imemory_ondisk: Don't fail write under any circumstances if locking is disabled (#2791)

* imemory_ondisk: Don't fail write under any circumstances if locking is disabled

* fmt

* inmemory_ondisk: Add a log message on failure

* clippy'

* micro optimization

* Revert changes to global Cargo.toml

* Hide std-dependent dependency behind std feature

* Fix example fuzzer

* Rename constructor for filtered fuzzer

* Reorder generics alphabetically

* Rename HashingMutator, add note to MutationResult about filtered fuzzers

* Improve StdFuzzer according to feedback

* rename hashing mutator

* Fix english in comment

* Cleanup of old PRs that break the CI

* Fix more CI bugs

* Code cleanup

* Remove unnecessary comments

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Sharad Khanna <sharad@mineo333.dev>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: s1341 <s1341@users.noreply.github.com>
2024-12-28 15:22:31 +01:00
Valentin Huber
25386bfb5f
Use #[expect(…)] instead of #[allow(…)], remove unnecessary allows (#2784)
* Use expect instead of allow, remove unnecessary allows

* Remove more whitelist lint warnings

* tranisitioning more subprojects

* Re-add some necessary allows

* Re-add more required allows

* Some more windows clippy fixes

* Re-add more whitelist items for expect

* More clippy whitelist fun

* Reset changes to generated files

* Reset generated files to correct version

* Move libafl_concolic to expect instead of allow

* Move libafl_frida to expect from allow

* Move libafl_libfuzzer to expect from allow

* Remove more whitelist items for macOS

* Fix unknown clippy allow

* Remove more unnecessary allow statements

* moving fuzzers

* Remove mistakenly added subdirs

* fixing imports

* Remove more unnecessary whitelisted lints

* Fix test for /home/ubuntu/LibAFL/fuzzers/inprocess/libfuzzer_libpng_accounting

* More clippy improvements for libafl_qemu

* fmt

* Some pedantic options

* Fix more stuff

* Remove Little-CMS again

* Add note to static_mut_refs

* Reset the changed testing routine since it is unnecessary
2024-12-20 16:55:35 +01:00
cube0x8
e46cf8a851
LibAFL_QEMU: Fix incorrect handling of brk syscall when shrinking the heap (#2776)
* added libafl_get_initial_brk API to properly handle brk growing and shrinking

* cargo fmt

* updated qemu revision

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-12-19 10:40:16 +01:00
a13dca6f39 abstract SystemTraceData 2024-12-16 16:00:18 +01:00
Dongjia "toka" Zhang
cc442f4dc8
No Uses* from fuzzer (#2761)
* go

* fixing stuf

* hello from windows

* more

* lolg

* lolf

* fix

* a

---------

Co-authored-by: Your Name <you@example.com>
2024-12-12 20:40:28 +01:00
8d7e32559f type fixing 2024-12-06 16:10:25 +01:00
1430532b38 fix build 2024-12-04 13:03:38 +01:00
c13537a620 WIP: port to 0.14.1 2024-12-03 15:32:17 +01:00
Scott
0dabc34329
Fix assertion in snapshot module for pages that are equal to SNAPSHOT_PAGE_SIZE (#2738)
Co-authored-by: Scott Powell <scott.powell@str.us>
2024-12-03 10:34:56 +01:00
Marco C.
36734083f9
Intel PT minor fixes/improvements (#2724)
* waitpid_filtered to ignore SIGWINCH

* Fix warnings unused manifest key: *.version

* Add export_raw feature to libafl_intelpt

* derive Debug for IntelPTHook

* Clippy

* Update target program ELF offsets

* Add comment to KVM pt_mode check

* refactor

* Add intel_pt_export_raw feature in libafl

* map_error instead of unwrap

* borrow checker friendly join_split_trace

and copy trace before deocde to prevent decoding failures

* Set ip_filters (also) with builder

* Move trace to file

* Fix Cargo.toml docs

* Ignore blocks with no instruction

most likely they are filtered out
2024-12-03 07:43:17 +00:00
Dongjia "toka" Zhang
95d87bd7d8
0.14.1 (#2698)
* 0.14.1

* fixer

* don't build nyx

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-12-02 21:06:29 +01:00
Romain Malmain
94fa4014ac
Update pyo3 to version 0.23.2 (#2732)
* update pyo3 to latest version

* add python bindings to workspace

* make pyo3 stuff dependent of workspace again

* adapt implementation for the newest version of pyo3
2024-11-27 19:01:31 +01:00
Romain Malmain
b3d3c38b29
Fix edge module generators (#2702)
* fix generators
2024-11-18 19:48:31 +01:00
Romain Malmain
b324e88631
Pre init module hooks (#2704)
* differenciate pre qemu init and post qemu init hooks

* api breakage: Emulator::new_with_qemu is not public anymore.
2024-11-18 19:47:14 +01:00
dependabot[bot]
b4a69d15d0
Update pyo3-build-config requirement from 0.22.3 to 0.23.1 (#2701)
Updates the requirements on [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.22.3...v0.23.1)

---
updated-dependencies:
- dependency-name: pyo3-build-config
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18 13:35:31 -03:00
Dominik Maier
b3d73b2919
Fix tui monitor for example fuzzers (#2699)
* Fix tui monitor for example fuzzers

* New clippy lint

* fix
2024-11-16 08:28:11 +01:00
Marco C.
eb2ac10d1e
Libafl workspace internal deps in workspace Cargo.toml (#2691)
* Add internal deps to workspace

* libafl: use workspace internal deps

* libafl_bolts: use workspace internal deps

* 0.14.0

* use workspace internal deps
2024-11-15 18:13:39 +01:00
Andrea Fioraldi
d339a58f7c
Bump to 0.14.0 (#2692) 2024-11-13 23:10:35 +01:00
Dominik Maier
7fada7d985
Add DrCovReader to read DrCov files and DrCov dumper and merge utils (#2680)
* Add DrCov Reader

* Removed libafl_jumper deps

* Fix DrCovWriter, add dump_drcov_addrs

* Taplo

* Move frida from usize to u64

* DrCov usize=>u64

* Better error print

* More u64

* ?

* debug

* clippy

* clippy

* Add Merge option to DrCovReader

* Add drcov_merge tool

* Move folder around

* DrCov

* More assert

* fmt

* Move around

* Fix print

* Add option to read multiple files/full folders
2024-11-12 22:18:57 -03:00
Dominik Maier
0ef0684e43
LibAFL_QEMU: Don't return a generic Address from Register reads (#2681)
* LibAFL_QEMU: Make ReadReg always return GuestReg type

* Don't return a generic address

* fix fuzzers

* fix mips
2024-11-12 22:14:35 -03:00
Marco C.
f3aa88b400
Use version.workspace (#2682) 2024-11-12 15:45:35 +01:00
jejuisland87654
4269be1c08
fix error '#' is not followed by a macro parameter (#2678) 2024-11-11 19:43:27 +01:00
Dominik Maier
8617fa6603
Introduce workspace (again) (#2673)
* Trying to redo workspace deps again after #2672

* unused

* clippy
2024-11-09 19:11:01 +01:00
Dominik Maier
21f8b1d147
Replace addr_of with &raw across the codebase (#2669)
* Replace addr_of with &raw across the codebase

* fix fixes

* more fix

* undo clang fmt?

* oops

* fix?

* allocator fix

* more fix

* more more

* more docs

* more fix

* mas mas mas

* hm

* more

* fix Frida

* needed

* more error

* qemu
2024-11-08 17:19:55 +01:00
dependabot[bot]
4581c50023
Update hashbrown requirement from 0.14.5 to 0.15.1 (#2660)
---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-05 14:46:40 +01:00
Dongjia "toka" Zhang
e0c5304e32
Don't use workspace dependencies (#2652)
* rev

* postcard

* serde

* fmt

* a

* dependabot

* postcard again

* hashbrown

* upd

* add

* serde?
2024-11-04 17:47:40 +01:00
dependabot[bot]
d7ce89d78d
Update goblin requirement from 0.8.2 to 0.9.2 (#2655)
Updates the requirements on [goblin](https://github.com/m4b/goblin) to permit the latest version.
- [Changelog](https://github.com/m4b/goblin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/m4b/goblin/commits)

---
updated-dependencies:
- dependency-name: goblin
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 16:23:27 +01:00
Romain Malmain
49ea0b03a6
Compile-time edge module compilation check, native support for ConstMapObserver (#2592)
* compile-time edge module compilation trick

* clippy

* possible since rust 1.79

* split edge module in submodules

* Update frida to 0.14.0 (#2596)

* update frida crate to the latest version

* adapt libafl_frida to the latest version of frida

* tracers and generators private modules

* do not use star export.

* same for drcov

* forgot a file...

* first draft of generic-based edge module for ConstantLengthMapObserver.

* integration of OwnedSizedSlice.

replaced OwnedSlice in ConstMapObserver by the new OwnedSizedSlice.

* fix serde stuff

* no std

* import

* fixed qemu_cmin with new constant map abstraction.

* fix const map

* fix clippy from another pr...

* fix non-null usage

* fix ci?

* new feature stuff

* fixes

* minor fixes

* fmt

* non null

* im stupid

* fmt

* fix fuzzer

* fix fuzzers

* sized slice

* fuzzer fixes

* ptr::NonNull -> NonNull

* shorter trait length

* fmt
2024-11-04 14:34:52 +01:00
Yufei Li
97a8e4c294
Add RISCV support in libafl_qemu.h (#2380)
* Add riscv support in libafl qemu header

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-11-04 11:10:25 +01:00
Romain Malmain
99a156f783
Implement From LibAFL Error for Qemu Error (#2641)
* impl from qemu error for libafl error

* new error for frontends like qemu or frida

* exhaustive match

* frontend -> runtime
2024-11-03 21:09:16 +01:00
Romain Malmain
6d55626a48
libafl_qemu: update qemu to v9.1.1 (#2636)
* update qemu to v9.1.1

* adapting stuff to qemu 9.1

* fix for new qemu gen_callN and x86 decoder

* remove outdated qemu configuration option
2024-10-31 17:54:37 +01:00
Romain Malmain
9da113e7a8
Fix RISC-V port issues (#2642)
fix riscv{32,64} stuff
2024-10-30 14:10:50 +01:00
saibotk
83c87acd5b
libafl_qemu: Add RISCV support (#2367)
* libafl_qemu: Add RISCV support

Adds the following targets (as features):
- riscv32
- riscv64

Added `RISCVCPU` and `CPURISCVState` to the bindings allow list.

Added riscv.rs to the arch module, with all necessary functions and
registers implemented and mapped.
The registers are the same as the ones found in qemus gdbstub xml found
after a build.

Additionally we added all syscall numbers for riscv 64 bit (already
supported by the `syscall_numbers` crate) and also added the missing
ones for riscv 32 bit. We compared both lists and their differences /
equalities with a simple python script and generated a list of the
missing ones, to be complete.
We might PR those to the `syscall_numbers` crate later on.

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-10-30 10:33:03 +01:00
Aarnav
0f744a3abb
Add Python Grammar Loader for Nautilus (#2635)
* add python grammar loader for Nautilus

* fmt

* fmt toml

* add python to macos CI deps

* install python

* fmt

* ci

* clippy

* fix workflow

* fmt

* fix baby nautilus

* fix nautilus sync

* fmt

* fmt

* clippy

* typo

* fix miri

* remove pyo3 from workspace to packages which need it and make it optional

* go back to AsRef<Path> for nautilus grammar loading

* replace hardcoded python flags for macos build

* typo

* taplo fmt

* revert formatting of libafl_qemu_arch

* ci

* typo

* remove expects in NautilusContext::from_file and make them Results

* remove not(miri) clause in test

* try and fix python build fir ios and android

* again

* android

* tmate

* fix android build

* document load_python_grammar

* log if python or json when loading nautilus grammar

* make nautilus optional

* add nautilus as feature to forkserver_simple_nautilus
2024-10-29 11:32:59 +01:00
Marco C.
b9e6363826
Avoid using feature flags and env variable to set the same parameter pt.1 emulation_mode (#2512)
* Remove emulation_mode env variable and custom cfg

* Using only the feature flag simplifies things a bit and allow the usage of optional dependencies

* Do not use --all-features on libafl_qemu

* Add missing target_os = "linux"
2024-10-25 15:16:55 +02:00
Romain Malmain
d96d833760
Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620)
* support (unsafe) on_crash / on_timeout callbacks for modules

* use libc types in bindgen
2024-10-21 17:59:04 +02:00
Romain Malmain
3b31b4d796
libafl_qemu: unset thumb bit for breakpoints (#2619)
* unset thumb bit for breakpoints
2024-10-17 09:42:56 +02:00
Romain Malmain
d48a7d508d
Centralize clippy lints in workspace (#2606)
* centralize clippy definition

* fmt

* add update bindings script

* add a checked and unchecked version of memory read to qemu stuff

also, a lot of clippy thing

* update binding position

* rm old script, new one is a bit better

* update doc

* macos clippy

* adapt fuzzers

* windows clippy

* fix fuzzer

* windows clippy

* remove old allowed clippy

* remove some allowed clippy

* use default features for serde_json in gramatron

* better error handler for failed rw to memory
2024-10-15 13:31:01 +02:00
Romain Malmain
18fb6a4197
[libafl-qemu-bridge] fix rpath regex for Ubuntu (#2609) 2024-10-11 18:56:51 +02:00
jejuisland87654
efa99ee3c9
Fix error[E0308]: mismatched types for libafl_qemu_init (#2593) 2024-10-09 17:17:51 +02:00
Dongjia "toka" Zhang
c12c6f31e2
Change fuzzbench_qemu fuzzer (#2520)
* change fuzzbench_qemu

* real test

* fix qemu crash hook

* update bindings

* fix fork executor, reduce trait bound overhead

* make EdgeModule depend on observer to get ptrs.

* do not make EdgeCoverageModule::new public

* map observer as builder call

* adapt examples with new edge coverage module builder.

* TMP: everyone is a variable length map observer

* reuse profile path script

* fix absolute paths

* remove some dependencies to make pipeline faster

* compile-time builder initialization check

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-10-08 15:18:13 +02:00
Romain Malmain
7344fdf059
Re-add drcov for both usermode and systemmode. (#2573)
* re-add drcov for both usermode and systemmode.
2024-10-07 14:20:11 +02:00
Dominik Maier
27677a6461
LibAFL_QEMU: Don't require extra_tokens. (#2576)
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-10-07 13:39:55 +02:00
Romain Malmain
1fbb18cb99
Versioning unification, dependencies update, logging optimization (#2560)
* versioning unification: use x.y.z format everywhere

* do not compile low-level logs (< info level) by default in fuzzers

* update dependencies to the latest versions

* add members to workspace.

* use workspace for common dependencies

* add vscode native support

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-10-07 12:03:29 +02:00
Romain Malmain
76d52bcf49
Addr filter update helper functions (#2575)
* add filter update helpers

* fmt
2024-10-02 14:26:46 +02:00
Dongjia "toka" Zhang
cbce7ffdfd
No Uses* for Feedbacks (#2578)
* add from addison's change

* fixing

* one left

* adding

* 444444

* okoko

* no prelude

* i hate rust

* stuff

* no std/win

* getting closer

* a

* a

* aa

* a

* a

* libafl_fuzz

* toml

* tutorial

* libafl_libfuzzer
2024-10-02 14:07:08 +02:00
Dongjia "toka" Zhang
a212d66afe
No Uses* for Observers (#2568)
* first

* Round 2

* Round 3

* Round 4

* Round 5

* Round 6

* Round 7

* Round 8

* clp

* from windows

* roll back

* baby building

* doc

* no deref

* frida

* api

* a

* impl HasLen

* libafl-fuzz and fmt

* asan error

* Cargo toml

* who cares abotu freebsd seriously?????

* aaaa

* qemu stuf

* fixing what i can fix

* nyx

* more for qemu stuf

* fixup libfuzzer

* fix baby swap

* AsIterMut

* UsesObservers eliminated

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Addison Crump <addison.crump@cispa.de>
2024-10-01 17:51:09 +02:00
Dominik Maier
82110472d6
Mark unsafe functions unsafe, as Ferris intended (#2559)
* Mark unsafe functions unsafe, as Ferris inteded

* More

* more safety?

* more fix

* actually safe

* More cleanup

* More fix

* more unsafe

* fix imports

* more unsafe

* fixes

* bring back the memories
2024-09-28 16:46:39 +02:00
Dongjia "toka" Zhang
79b8390214
No Uses* for Corpus, Mutators (#2547)
* wip

* this shit is crazy

* from windows

* fmt

* fuck

* fmt

* aa

* reduce generics for generator

* do stuff for mutators, too

* aa

* a

* fmt

* idk

* getting things compiling

* fix merge

* idk

* fuzzers

* nire

* more

* a

* frida win

* cmin

* mm

* qemu?

* del ubuntu dep

---------

Co-authored-by: Your Name <you@example.com>
2024-09-26 20:03:03 +02:00
Romain Malmain
c944a70056
Linux kernel fuzzing example (#2496)
* linux kernel (x509_cert) and process fuzzing example

* rework filters

* update to latest qemu

* working for process and kernel fuzzing

* new i2s mutator for binary only fuzzers

* refactoring modules with new filtering interface

* add state as parameter of harness

* hide unused global in usermode

* Script for stub bindings generation

* do not try to check whether it is worth generating the bindings, always
  generate when the env variable is on.

* add taplo to fmt_all.sh

* Moved fuzzers (again) in a target-centric way.

* fix rust 2024 warnings.

* new libafl_qemu harness structure.

* rename qemu_systemmode into qemu_baremetal

* fix qemu baremetal makefile

* fix formatter

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-09-26 14:29:33 +02:00