1314 Commits

Author SHA1 Message Date
Dhanvith Nayak
72986fc129
Deduplicate OnDisk Corpus (#2827)
* testcase name logic

* implement locking

* implement logic for removing testcase

* minor modifications

* minor modifications to remove_testcase()

* extract generate_name() from trait Input (broken)

* Revert "extract generate_name() from trait Input (broken)"

This reverts commit 9e217be2810a8c33970846cdd380f542975cc05b.

* fix ci errors

* remove CorpusId from generate_name() calls

* toml formatting

* write from file instead of fs

* fmt and clippy

* fix windows clippy

* handle renaming of testcase

* fix failing cmplog test

* overwrite lockfile on remove testcase

* format

* bring back corpus id in generate_name

* missed windows executors hook

* fix failing tests

* some more errors

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2025-02-04 07:41:00 +01:00
Valentin Huber
500e01816d
Rename BoolMutator to BoolInvertMutator (#2929)
* Rename BoolMutator to BoolInvertMutator

* Fix name of BoolInvertMutator
2025-02-03 13:33:39 +01:00
Valentin Huber
6243b684f8
Introduce BoolMutator (#2926) 2025-02-01 21:24:24 +01:00
Dominik Maier
84702d12a8
Make sure EM and Z remain consistent in InProcessExecutor (#2873)
* Make sure EM and Z remain consistent in InProcessExecutor

* first make them compile

* a

* hah

* fix intel pt

* decouple fuzzer from em

* lol

* 3

* fix tcp

* fix

* fix

* fix

* fixer

* std

* fixer

* plz

* plzplzplz

* plzplzplzplz

* mm

* more

* symbol

* a

* a

* mm

* mmm

* mmmm

* mmmmm

* ff

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2025-02-01 08:23:40 +01:00
Dongjia "toka" Zhang
6cd97e7105
Decouple fuzzer functions from event manager (#2915)
* decouple fuzzer from em

* lol

* 3

* fix tcp

* fix

* fix

* fix

* fixer

* std

* fixer

* plz

* plzplzplz

* plzplzplzplz

* mm

* more

* symbol

* a

* a

* mm

* mmm

* mmmm

* mmmmm

* ff
2025-02-01 07:52:42 +01:00
Railroad6230
85c1d03425
Rename libafl_bolts::rands::Rand::zero_upto to below_or_zero. (#2911)
Hi LibAFL!

I was playing with the [`Rand`] trait when I realized that the
documentation of [`Rand::zero_upto`] did not match what I was expected:

fd6271fa35/libafl_bolts/src/rands/mod.rs (L139-L142)

When using the following RNGs, [`Rand::zero_upto`] never returns the upper bound `n` as it would have been expected according to the documentation:

 - `RomuDuoJrRand`
 - `RomuTrioRand`
 - `Sfc64Rand`
 - `XkcdRand`
 - `XorShift64Rand`
 - `Xoshiro256PlusPlusRand`

The default implementation of [`Rand::zero_upto`] is to use [`fast_bound_usize`],
which excludes the given upper bound, thus I believe here that the default implementation
of [`Rand::zero_upto`] is wrong.

As discussed here: https://github.com/AFLplusplus/LibAFL/pull/2911#issuecomment-2623773829,
we believe that renaming the method would be better than changing the actual
implementation.

[`Rand`]: fd6271fa35/libafl_bolts/src/rands/mod.rs (L108)
[`Rand::zero_upto`]: fd6271fa35/libafl_bolts/src/rands/mod.rs (L139-L142)
[`fast_bound_usize`]: fd6271fa35/libafl_bolts/src/rands/mod.rs (L100-L103)
2025-01-30 14:16:23 +01:00
Dongjia "toka" Zhang
d8df9b4910
Fix Clippy (#2913)
* clp

* lol

* this is false positive

* a

* clippy all

* aa
2025-01-30 11:17:01 +01:00
Dongjia "toka" Zhang
fd6271fa35
Delete useless trait bounds from share_objectives feature (#2908)
* fix

* a
2025-01-29 13:55:38 +01:00
Dongjia "toka" Zhang
5002336fad
Delete evaluate_input_events (#2906) 2025-01-28 19:06:43 +01:00
EvianZhang
f30b05419b
Add combined monitor (#2900)
* Add combined monitor

* Sync start_time when initializing

* Use clone_from instead of clone to make clippy happy
2025-01-28 12:13:41 +01:00
Dongjia "toka" Zhang
133a0ffe7a
Merge LlmpEventManager and LlmpRestartingEventManager (#2891)
* add

* add 2

* feature

* fix nyx launcher

* a bit of doc

* addressing comments
2025-01-26 13:43:04 +01:00
Romain Malmain
4083f0ba73
Fix drcov path parsing (#2884)
* fix drcov path parsing

* refactoring of drcov tool

* add the possibility to sort addresses in drcov tools

* more aggressive clippy. it now catches more warnings as errors than before

* reduce the number of unfixable warnings displayed.
2025-01-24 18:21:51 +01:00
Dongjia "toka" Zhang
2842f95e2a
Dedup StdOut/StdErr observer pt.2 (#2876)
* dedup stdout stdin

* mm

* Claude was not enough

* but was good enough

* ok

* okok

* okokok

* shorter now

* lol

* fixer
2025-01-21 16:53:02 +01:00
Dongjia "toka" Zhang
6eabb79920
Dedup stdout/stdin observer (#2871)
* dedup stdout stdin

* mm

* Claude was not enough

* but was good enough

* ok

* okok

* okokok

* shorter now
2025-01-21 15:17:07 +01:00
Dominik Maier
7e18887a32
Remove shmem associated type (#2870)
* reduce shm trait bound

* Rename to SendExiting

* alpha beta gamam

* alphabet

* work

* std only

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2025-01-20 20:28:19 +01:00
Romain Malmain
8089b18d34
Remove ShMemProvider bound from struct definitions (#2861)
* No more ShMemProvider bound constraint in struct definition whenever possible

* Introduce StdShMem

* Update CONTRIBUTING.md
2025-01-20 17:25:55 +01:00
Dominik Maier
a949452453
Rename ResizableMutator, make it generic (#2859)
* Rename ResizableMutator, make it generic

* fix

* more fix

* reorer

* fmt, fixes

* fixes

* more commit

* fmt
2025-01-19 17:17:50 +00:00
Dhanvith Nayak
2a36b78fd6
Share objectives between nodes (#2754)
* add support to share new objectives in CentralizedEventManager

* handle received Objectives

* remove duplicate event fires in centralized event manager

* share input on share_objectives feature (broken)

* split impl LlmpEventManager based on share_objectives

* reduce code duplication in impl LlmpEventManager (broken)

* fix traits error (temp)

* fix mismatched types

* fix cargo format issue

* merge duplicated functions into single impl in llmp/mod.rs

* merge duplicate impl blocks in stages/sync.rs

* fix clippy warnings

* deduplicate handle_in_client

* cleanup unnecessary code

* handle objectives in tcp eventmanager

* handle objectives in llmp eventmanager (broken)

* handle objectives in llmp eventmanager

* fix doc test

* format

* clippy

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2025-01-19 17:00:24 +01:00
Dongjia "toka" Zhang
2e26af90db
Better error message in testcase_score.rs pt.2 (#2863)
* msg

* l
2025-01-18 13:07:14 +01:00
Dongjia "toka" Zhang
019c71a368
Better error for missing TopRatedsMetadata (#2860) 2025-01-17 21:42:51 +01:00
Romain Malmain
f8ad61e14a
Remove Input associated type (#2854)
* Completely remove Input as an associated type in multiple traits

* Unify usage of Input as generic instead 

* Remove many unused bounds, in particular HasCorpus

* fix multiple generic ordering

* update and fix CONTRIBUTING.md

* update MIGRATION

* use the same generic input type for new / with_max_iterations to make typing easier in most cases.

* Restore libafl_libfuzzer test in CI
2025-01-17 14:53:51 +01:00
Valentin Huber
93c5adde4d
Remove MapObserver dependency from observer-dependent stages and schedulers in favour of generic hashing (#2851)
* Introdue SimpleHash separate from MapObserver

* Move to Hash for hashing

* Fix docs, remove even more restrictions

* fix libafl_targets

* fix fuzzer

* Remove broken and unnecessary derive

* Remove unnecessary trait restriction

* Remove unnecessary import

* Add changes to MIGRATION.md

* Remove more unnecessary imports
2025-01-16 16:34:58 +00:00
Railroad6230
15aa498d5e
Fix #2853: split HasMutatorBytes trait into two traits. (#2856)
(See #2853.)

This commit attempts to improve the [`HasMutatorBytes`] trait to allow it to
be used along with input types that cannot be resizable.

[`HasMutatorBytes`] is split into two traits:

 - [`HasMutatorBytes`]: requires `bytes` and `bytes_mut`.
 - `HasMutatorResizableBytes`: requires [`HasMutatorBytes`], and also requires
   a few other methods for resizing / shrinking the underlying input type.


N.B.: I believe that if merged, this would introduce a breaking change.

[`HasMutatorBytes`]: 198cd5dbc5/libafl/src/inputs/bytes.rs (L26)
2025-01-16 17:06:19 +01:00
bernhl
c9531aa242
Remove unecessary HasTargetBytes constraint (#2852) 2025-01-16 10:51:18 +01:00
Dongjia "toka" Zhang
435ca021cc
No uses for EventManager (#2831)
* the first step of the last step

* wip

* 99% done

* 99.9% done

* 99.99

* Hello from windows

* aaa

* 99.999

* aa

* 1

* 2

* 3

* 4

* 5

* plz

* plzplzplz

---------

Co-authored-by: Your Name <you@example.com>
2025-01-15 16:56:40 +01:00
Valentin Huber
ba09cb0706
Add OnDiskJsonAggregateMonitor (#2845)
* Add OnDiskJsonAggregateMonitor

* Fix formatting

* Remove unnecessary trait bounds

* Remove unnecessary import restriction

* Remove unnecessary imports

* fix formatting
2025-01-15 13:40:26 +01:00
Romain Malmain
a45e44764f
Remove UsesInput from libafl qemu (#2832)
* remove UsesInput from libafl_qemu

* Insert I bound instead of ugly Corpus::Input stuff

* Uses a C generic for Commands

* adapt nyx api to new generics

* add qemu linux fuzzers to CI for building.
2025-01-14 10:24:28 +01:00
Dominik Maier
02566b33cd
Make Stats stage send stats again (#2830)
* Make Stats stage send stats again

* re-remove stats mod

* clp, fmt

* clip
2025-01-13 15:06:26 +00:00
Dongjia "toka" Zhang
fd06e5ced0
Remove CustomBufHandlers (#2829)
* rem

* fix

* fixer
2025-01-13 16:00:41 +01:00
Dongjia "toka" Zhang
aa0391ef8d
No Uses* from executor (#2766)
* first step

* wip

* lol

* making progress, slowly

* add

* a bit more

* progress

* 80%done

* 90% done

* done from linux

* Hello from windows

* 1

* 2

* 3

* 4

* 5

* 6

* 7

* Hello from windows

* 9

* 10

* 11

* 11

* 12

* 13

* aa

* fixer

* a

* oops

---------

Co-authored-by: Your Name <you@example.com>
2025-01-11 15:25:15 +01:00
Dominik Maier
f7745155c9
Upgrade all Windows crates (#2823)
* Upgrade all win crates

* wtf

* fix
2025-01-09 00:25:33 +01:00
Valentin Huber
9b8b552f27
Make more int mutator types public (#2821) 2025-01-07 17:02:13 +01:00
Dominik Maier
719a3c0f07
Continue loading inputs even if some fail to deserialize (#2820)
* Continue loading inputs even if some fail to deserialize

* fmt

* fix
2025-01-07 15:07:46 +01:00
Romain Malmain
7c8708d4b1
Refactor of Qemu configuration (#2707)
* Qemu config refactoring.

* QEMU error refactoring.

* Single QEMU init function.

* Light refactor of EmulatorModules.

* Qemu is now a parameter to EmulatorModule callbacks and most function hooks.

* EmulatorModules is initialized before QEMU is initialized.

* refactor asan and asanguest modules to avoid custom init of QEMU and use the module interface instead.

* asan fixed size accesses working with generics.

* use pre_syscall_* and post_syscall_* everywhere for consistency.

* adapt qemu_launcher example to fully work with Emulator, since Qemu must now be initialized by Emulator.

* start writing Emulator / EmulatorBuilder / QemuConfig doc.

* fix broken intel pt doc.
2025-01-06 15:04:40 +01:00
Dominik Maier
742773bc17
Add BoolValueFeedback (#2815)
* Add BoolValueFeedback

* No_std

* clippy

* Fix tests

* More clip

* fix no_std tests
2025-01-06 04:25:36 +01:00
Dominik Maier
4b4a22bc44
Cargo Format (#2814)
* Initial commit: ValueBloomFeedback

* Add test, fix feedback

* Remove unneeded feedback

* fix

* more commit

* Cargo fmt

* fmt
2025-01-06 02:11:47 +01:00
Dominik Maier
2a3f5a5942
Add Bloomfilter-based Feedback for Values (#2813)
* Initial commit: ValueBloomFeedback

* Add test, fix feedback

* Remove unneeded feedback

* fix

* more commit
2025-01-06 02:03:18 +01:00
Dominik Maier
d39ded5b29
Fix and/or mute more clippy lints, deprecation warnings in Pyo3 (#2805) 2025-01-03 15:17:46 +01:00
Dominik Maier
deb76555b7
New year new clippy (#2797)
* New year new clippy

* More clipy

* fix
2025-01-01 20:57:43 +01:00
AshrafIbrahim03
92db678995
Added expect error message to TimeFeedback where there used to be an unwrap (#2777)
* Added expect error message to TimeFeedback where there used to be an unwrap

* Changed error message handling

* fixed cargo clippy

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-12-30 20:47:04 +01:00
Mehtab Zafar
8cd069cf3e
Optimize event serialization with pre-allocated buffer (#2794)
* Optimize event serialization with pre-allocated buffer

- Added event_buffer field to LlmpEventManager
- Used to_slice instead of to_allocvec
- Pre-allocated buffer size is 4KB

Fixes #1082

* Fallback to to_allocvec in case of event_buffer overflow

Also combined the shared logic between compressed & uncompressed event
firing while keeping the same behavior

* Made the initial event_buffer size to a const

Also removed the unnecessary event_buffer.clear(), since we are already
resizing it
2024-12-30 20:45:48 +01:00
Valentin Huber
d8ec991b48
Add bloom filter for duplicate execution of the same inputs (#2771)
* fixing empty multipart name

* fixing clippy

* New rules for the contributing (#2752)

* Rules

* more

* aa

* Improve Flexibility of DumpToDiskStage (#2753)

* fixing empty multipart name

* fixing clippy

* improve flexibility of DumpToDiskStage

* adding note to MIGRATION.md

* Update bindgen requirement from 0.70.1 to 0.71.1 (#2756)

Updates the requirements on [bindgen](https://github.com/rust-lang/rust-bindgen) to permit the latest version.
- [Release notes](https://github.com/rust-lang/rust-bindgen/releases)
- [Changelog](https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/rust-bindgen/compare/v0.70.1...v0.71.1)

---
updated-dependencies:
- dependency-name: bindgen
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* No Use* from stages (#2745)

* no from stage

* fixer

* doc fix

* how was this working????

* more fixes

* delete more

* rq

* cargo-fuzz

* m

* aa

* Update CONTRIBUTING.md MIGRATION.md (#2762)

* No Uses* from `fuzzer` (#2761)

* go

* fixing stuf

* hello from windows

* more

* lolg

* lolf

* fix

* a

---------

Co-authored-by: Your Name <you@example.com>

* Remove useless cfgs (#2764)

* Link libresolv on all Apple OSs (#2767)

* Somewhat ugly CI fix... (#2768)

* Maybe fix CI

* does this help?

* Very dirty 'fix'

* Add Input Types and Mutators for Numeric Types (#2760)

* fixing empty multipart name

* fixing clippy

* New rules for the contributing (#2752)

* Rules

* more

* aa

* Improve Flexibility of DumpToDiskStage (#2753)

* fixing empty multipart name

* fixing clippy

* improve flexibility of DumpToDiskStage

* adding note to MIGRATION.md

* Introduce WrappingMutator

* introducing mutators for int types

* fixing no_std

* random fixes

* Add hash derivation for WrappingInput

* Revert fixes that broke things

* Derive Default on WrappingInput

* Add unit tests

* Fixes according to code review

* introduce mappable ValueInputs

* remove unnecessary comments

* Elide more lifetimes

* remove dead code

* simplify hashing

* improve docs

* improve randomization

* rename method to align with standard library

* add typedefs for int types for ValueMutRefInput

* rename test

* add safety notice to trait function

* improve randomize performance for i128/u128

* rename macro

* improve comment

* actually check return values in test

* make 128 bit int randomize even more efficient

* shifting signed values

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>

* Add HashMutator

* Fix docs

* Fix docs again

* introducing bloom filter

* fix tests

* Implement evaluate_filtered

* Add macros to libafl_bolts tuples for mapping and merging types (#2788)

* Add macros

* Use the macros for havoc_mutations

* Fix docs

* improve merge_tuple_list_type to accept n items

* libafl_cc: Automatically find llvm_ar path (#2790)

* imemory_ondisk: Don't fail write under any circumstances if locking is disabled (#2791)

* imemory_ondisk: Don't fail write under any circumstances if locking is disabled

* fmt

* inmemory_ondisk: Add a log message on failure

* clippy'

* micro optimization

* Revert changes to global Cargo.toml

* Hide std-dependent dependency behind std feature

* Fix example fuzzer

* Rename constructor for filtered fuzzer

* Reorder generics alphabetically

* Rename HashingMutator, add note to MutationResult about filtered fuzzers

* Improve StdFuzzer according to feedback

* rename hashing mutator

* Fix english in comment

* Cleanup of old PRs that break the CI

* Fix more CI bugs

* Code cleanup

* Remove unnecessary comments

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Sharad Khanna <sharad@mineo333.dev>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: s1341 <s1341@users.noreply.github.com>
2024-12-28 15:22:31 +01:00
s1341
9b4cd51c63
imemory_ondisk: Don't fail write under any circumstances if locking is disabled (#2791)
* imemory_ondisk: Don't fail write under any circumstances if locking is disabled

* fmt

* inmemory_ondisk: Add a log message on failure

* clippy'

* micro optimization
2024-12-24 14:22:01 +01:00
Valentin Huber
54202c3ef3
Add macros to libafl_bolts tuples for mapping and merging types (#2788)
* Add macros

* Use the macros for havoc_mutations

* Fix docs

* improve merge_tuple_list_type to accept n items
2024-12-23 13:25:14 +01:00
Valentin Huber
03f7fc93ad
Fix empty multipart (#2789) 2024-12-23 11:39:51 +01:00
Valentin Huber
25386bfb5f
Use #[expect(…)] instead of #[allow(…)], remove unnecessary allows (#2784)
* Use expect instead of allow, remove unnecessary allows

* Remove more whitelist lint warnings

* tranisitioning more subprojects

* Re-add some necessary allows

* Re-add more required allows

* Some more windows clippy fixes

* Re-add more whitelist items for expect

* More clippy whitelist fun

* Reset changes to generated files

* Reset generated files to correct version

* Move libafl_concolic to expect instead of allow

* Move libafl_frida to expect from allow

* Move libafl_libfuzzer to expect from allow

* Remove more whitelist items for macOS

* Fix unknown clippy allow

* Remove more unnecessary allow statements

* moving fuzzers

* Remove mistakenly added subdirs

* fixing imports

* Remove more unnecessary whitelisted lints

* Fix test for /home/ubuntu/LibAFL/fuzzers/inprocess/libfuzzer_libpng_accounting

* More clippy improvements for libafl_qemu

* fmt

* Some pedantic options

* Fix more stuff

* Remove Little-CMS again

* Add note to static_mut_refs

* Reset the changed testing routine since it is unnecessary
2024-12-20 16:55:35 +01:00
Celian G.
09950ccc34
Update last found time when evaluating an input (#2782)
Co-authored-by: celian <cglenaz>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2024-12-19 14:51:59 +01:00
Valentin Huber
5d70216cc7
Remove MutVecInput and MappedInput in Favour of Impls on References (#2783)
* Remove MutVecInput and MappedInput

* Rename mapping mutators

* Update MIGRATION.md

* Fix test in docs

* Rename mapping mutators mappers

* Fix MIGRATION.md

* Fix docs link
2024-12-19 14:35:17 +01:00
cube0x8
df3384d868
Separate Prometheus metrics into global and per-client categories with refactoring (#2781)
* separated global from per-client stats. refactoring

* cargo +nighlty fmt

* derive(Default) for PrometheusStats
2024-12-19 10:38:43 +01:00
Valentin Huber
57cecca927
Remove outdated doc (#2778) 2024-12-18 16:53:14 +01:00