3576 Commits

Author SHA1 Message Date
Josef Haider
d34965192d
Add FeedbackFactory implementations for CrashFeedback, CombinedFeedback and DiffFeedback. (#2060)
* Add FeedbackFactory implementations for CrashFeedback, CombinedFeedback and DiffFeedback

* remove redundant type CrashFeedbackFactory
2024-04-23 15:48:52 +02:00
Gregory Petrosyan
e1b8c9b5d8
libafl_bolts: improvements to the rands module, add next_float (#2086)
* rands: use splitmix64 for seeding

Seeding with splitmix64 is a good way to avoid starting with
low-entropy PRNG states, and is explicitly recommended
by the authors of both xoshiro256++ and Romu.

While at it, give the xoshiro256++ PRNG its proper name.

* rands: use fast_bound() to generate number in range

* rands: add top-level choose()

* rands: add Rand::next_float()

* rands: add Rand::coinflip() helper

* libafl: unbreak tests that relied on direct seeding

* rands: add SFC64 PRNG

SFC64 is a well-established and well-understood PRNG designed by
Chris Doty-Humphrey, the author of PractRand. It has been tested
quite a lot over the years, and to date has no known weaknesses.

Compared to xoshiro256++, it is slightly faster and is likely to
be a more future-proof design (xoshiro/xoroshiro family of generators
come with quite long history of [flaws][1] found over the years).

Compared to Romu, it is slightly slower, but guarantees absense
of bias, minimum period of at least 2^64 for any seed, and
non-overlapping streams for different seeds.

[1]: https://tom-kaitchuck.medium.com/designing-a-new-prng-1c4ffd27124d
2024-04-23 15:37:28 +02:00
Dongjia "toka" Zhang
5ff709f241
Symcc Update (#2047)
* symcc upd

* upda

* fix

* fmt

* mm

* psu

* psh

* fix

* fix

* fix

* tmate

* can't make it work

* i really hate you llvm

* real name

* mm
2024-04-23 13:34:46 +02:00
Romain Malmain
2046cfe82d
Checkout before filter (#2092) 2024-04-23 09:45:45 +02:00
Dongjia "toka" Zhang
ea6a36b0a5
fix (#2091) 2024-04-22 21:55:55 +02:00
Dongjia "toka" Zhang
0f3ad288e7
Fix #1932 (#2089)
* fix

* Stupid lint

* increase rate

* fix
2024-04-22 19:27:13 +02:00
Romain Malmain
98863fbff5
Only run QEMU tests if there is a QEMU-related change. (#2090)
* Only run QEMU tests if there is a change.

* fmt.
2024-04-22 19:14:23 +02:00
Romain Malmain
fef6c8d1b1
Run qemu_systemmode tests with self-hosted runners (#2018)
* run qemu fuzzers (qemu_systemmode only for now) in self-hosted runners

* Remove qemu-related fuzzers to general fuzzers

* fix

* Install dependencies before anything else

* Do not use sudo

* Install sudo

* Revert "Install dependencies before anything else"

This reverts commit 107addad5d9f68dec5a9af50831112cd72c28f4d.

* added qemu specific prerequisites

* add -y flag

* Format with nightly

* Do not use nightly only.
Install fmt and clippy for stable as well.

* Install qemu-img for qemu

* fix qemu-img install

* apt update

* Changed timeout.

* Fix qemu_systemmode test

* fmt

* clippy + decorrelate build and run for qemu_systemmode.

* fix fuzzer

* clippy

* add sqlite3-dev to package prerequisites.

* add arm-none-eabi-gcc

* fix profile dir

* fix condition.

* Run less QEMU stuff faster

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-04-22 18:17:22 +02:00
Tobias Mayer
fe64d10a5c
Fix typo in setup_libxml2.sh (#2088)
exit for a non-zero exit code
2024-04-22 14:37:41 +02:00
Dongjia "toka" Zhang
2df71e7a36
Faster cargo check (#2084)
* speed up cargo check

* fix

* composite

* plural

* wrong name

* go brrrrrrrr

* bash

* aaaaa

* why not even more?

* more composite

* fix

* miri

* test for llvm >= 16

* v2

* v2

* naaaa

* JUST MORE INSTANCES

* aa

* fix

* no bash for windows

* reduce a bit as some guys are delayed

* no deps
2024-04-20 10:50:43 +02:00
Addison Crump
fd364d15fe
cargo-fuzz testing (#2081)
* Build cargo-fuzz end to end

* Workaroudn failing CI

* Fix toml

* Build

* Exclude from workspace

* Fix version

* Fix workspace

* Fix workspace

* FIx workspace

* Fix path

* Set path correctly

* Install llvm-tools

* Test with distro llvm installed

* Use build

* complete

* remove from includes

* add back fuzzer tests, format

* add test for whether all fuzzers are actually tested in pipeline

* whoops

* I hate CI I hate CI I hate CI

* revert removal of llvm

* sh compatibility

* sigh

---------

Co-authored-by: Max Ammann <max@maxammann.org>
2024-04-19 23:55:28 +02:00
Dongjia "toka" Zhang
ea549b9cfe
Add a warning to say never use EDGES_MAP.len() (#2083)
* Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2…"

This reverts commit c68b3816fb680b635f99c337f78185e699864705.

* all

* more

* add

* add
2024-04-19 23:26:00 +02:00
Dongjia "toka" Zhang
e8cc1ba93d
Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use" (#2082)
* Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2…"

This reverts commit c68b3816fb680b635f99c337f78185e699864705.

* all

* more
2024-04-19 19:03:58 +02:00
edwin1729
c238b69498
Implement Hash for MapObserver (#1989)
* MapObserver implements Hash

* Rename the hash utility function (in MapObserver) to hash_easy

* Use hash_slice as a helper function to impl Hash trait

* define_python_map_observer macro implements Hash trait

* Also rename hash_easy to hash_simple

* Rename hash_slice to hash_helper

* hash_helper is used to define the implementation of hash function/trait

* Factor out the Hash trait and function for runtime library structs (#1977)

* Simplify hash_simple (of trait MapObserver) (#1977)

 * Use hash_one function to make hash_simple a one-liner

* remove hash_helper

---------

Co-authored-by: Edwin Fernando <ef322@ic.ac.uk>
Co-authored-by: Addison Crump <addison.crump@cispa.de>
2024-04-19 15:06:14 +02:00
Addison Crump
04cd792df2
fix libfuzzer by simply adding the needed feature (#2080) 2024-04-19 13:35:35 +02:00
Dongjia "toka" Zhang
c68b3816fb
Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2079)
* fix

* aa

* push

* fix

* fmt
2024-04-19 13:24:25 +02:00
Tobias Mayer
d7ac802fb2
Update libafl_nyx to newest qemu-nyx and nyx-packer, fixing some problems with build_nyx_support.sh. Providing Makefile for bug in qemu-nyx (#2070)
Co-authored-by: ganbarutobi <ganbarutobi@gmail.com>
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-04-19 10:41:55 +02:00
Addison Crump
276bfc6386
tee not pipe (#2078) 2024-04-18 18:23:57 +02:00
Romain Malmain
84a432dab2
fix import (#2076) 2024-04-18 15:43:21 +02:00
mkravchik
2b18562b08
Fixing the inconsistency btw #2012 and #2066 (#2074) 2024-04-18 13:33:29 +02:00
Romain Malmain
39c32d55e7
Fix main not passing CI tests (#2075)
* Fix sancov import

* fmt

* fmt
2024-04-18 13:33:18 +02:00
mkravchik
886519b10c
Pr/fixing forkserver libafl cc (#2066)
* Fixing forserver_libafl_cc

* Adding tests and showing user stats

* Restoring the map truncation

* Fmt

* small fix

* fix

* fix

* fix

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2024-04-17 18:28:34 +02:00
Dongjia "toka" Zhang
c50af44099
Split edges_map_size and edges_map_size_alloc (#2012)
* two sizes

* rename

* build.rs

* rename

* capital

* FMT

* macro

* fuck

* fff
2024-04-17 17:58:08 +02:00
Addison Crump
467c6d7582
missed a spot (#2063) 2024-04-17 16:24:15 +02:00
Dongjia "toka" Zhang
7595b2e5c9
Debugging tips upd (#2062) 2024-04-17 16:01:33 +02:00
Addison Crump
edcff3889c
fixup: fix some missing tracking hints (#2058) 2024-04-17 14:34:06 +02:00
Dongjia "toka" Zhang
8bce605503
DDFuzz (#2056)
* push

* add

* book

* ddg

* works
2024-04-16 16:51:28 +02:00
Romain Malmain
bc3ef5952b
Rework of libafl_qemu configuration (#2054)
* LibAFL QEMU can now be dynamically linked
* LibAFL QEMU reconfiguration happens less frequently (now using a signature check)
* Possibility to have custom rpath in QEMU
2024-04-16 11:35:15 +02:00
Dongjia "toka" Zhang
da6118e61e
Debugging tips (#2048)
* push

* add

* book
2024-04-15 13:40:54 +02:00
David CARLIER
4ec50d6a10
bolts: core affinity simplification for freebsd (#2049)
since 13.1 (EOL since mid-2023) , supports the sched affinity api like linux/dragonflybsd.
2024-04-14 10:29:32 +02:00
Dongjia "toka" Zhang
0c68ff033f
Readme Update (#2046) 2024-04-12 23:29:23 +02:00
Dongjia "toka" Zhang
cbb323f26c
LibAFL 0.12 (#1823) 0.12.0 2024-04-12 18:42:14 +02:00
Addison Crump
3d702f403b
Implement CanTrack tracking enforcement through rust types (#1886)
* sample implementation of tracking enforcement (incomplete)

* helpful compiler output

* make it look like a real compiler output

* ensure that the macro may be used outside of libafl

* separate index/novelty tracking funcs

* default const generic values so that we don't need to change this everywhere

* fix tests

* rollback unnecessary specification of stdmapobserver

* register metadata in doc tests

* doc fixes

* doc cleanup

* doc cleanup 2

* reduce implementor overhead to zero

* renaming/docs fixes

* asref isn't reflexive??

* generalization stage updates

* add better documentation about require_{indices,novelties}_tracking

* remaining generic updates

* round one CI pass (knowingly introduces breaking changes)

* typo

* round 2 clippy

* rollback: libafl_frida changes

* fmt

* moar porting

* fix remaining fuzzers

* fix windows build, maybe

* fixup libafl_libfuzzer

* fmt nighlty all the things

* attempt to fix some broken additions

* fix fmt

* oops

* fix new invocation

* minimizer scheduler fixes

* fix accounting

* rename

* fix

* Fix build

* Sort generics

* Move more generics into the right place

* Rename A -> C

* Fix test

* Fix test some more

* Fix doc some more

* critical formatting

* More A->C

* CanTrack harder

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-04-12 18:33:52 +02:00
Romain Malmain
ec935bf95f
Fix GuestMaps iterator in LibAFL QEMU. (#2041)
* Fix maps iterator.

* Use native QEMU structs instead of pointer casting.

* Update stub bindings.

* Maps operations stored in usermode.
2024-04-12 14:40:53 +02:00
Maurice
8b9b5a8767
Introduce StdOutToMetadataFeedback and StdErrToMetadataFeddback (#2042)
* introduce `StdOutFeedback` and `StdErrFeddback`

* hide behind "std" feature flag

* update

* update
2024-04-12 14:26:47 +02:00
Dominik Maier
2f618b62d7
Rename HasAFLSchedulerMetadata to simply AflScheduler (it doesn't hold a Metadata at all) (#2043) 2024-04-12 13:24:00 +02:00
Dominik Maier
0f778b5892
Order differential generics alphabetically (#2044) 2024-04-12 12:14:49 +02:00
Maurice
631b1746e2
libafl_nyx: Add support for StdOutObserver (#2033)
* libafl_nyx: Add support for `StdOutObserver`

* reset file offset

* update example fuzzers

* fix
2024-04-11 18:24:32 +02:00
Romain Malmain
94a2a2363a
Add CPUArchStatePtr to backdoor signature (#2038) 2024-04-11 17:30:59 +02:00
Dongjia "toka" Zhang
287d1ac7c7
Add HasAdaptiveSerializer trait. (#2040)
* fix

* revert test

* add

* a

* check
2024-04-11 15:36:08 +02:00
Stefan Zabka
48463d079b
Fix error handling in libafl_qemu_build (#2036)
* fix(libafl_qemu_build): assert command success

* fix(libafl_qemu_build): make sure linker_interceptor.py picks up correct compiler

Currently linker_interceptor.py uses 'cc' as the __LIBAFL_QEMU_BUILD_CC environment variable is never set

* remove redudant arg
2024-04-10 18:36:36 +02:00
Dongjia "toka" Zhang
0d5c6219d8
Dedup CentralizedEventManager's serialize_observers() (#2034)
* why

* clp
2024-04-10 16:13:03 +02:00
WorksButNotTested
374f8735fa
Implement user-space QEMU ASAN (#1806)
* Implement user-space QEMU ASAN

* Fix wrong cfgs

* fmt

* merge conflicts in libafl qemu

* A few more fixes to qemu_launcher

* Change commit of qemu-libafl-bridge

* Fix clippy in qemu_launcher

* Fix commit id again

* Empty commit to trigger CI

* Fix path to fuzzer for test in qemu_launcher?

* Revert location of target binary and show the full error log from qemu_launcher test

* Appease the clippy gods

* Empty

* Fix format

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2024-04-10 01:11:28 +02:00
Aarnav
47c41c2925
Introduce disabled testcases for splicing (#1932)
* introduce disabled field to Testcase

* separate executor's processing of execution (adding to corpus/solution/discarding) and execution of input

* introduce add_disabled_input function

* enable splicing mutators to fetch disabled inputs

* reset modified example

* clean up

* update docs

* update docs for count_with_disabled

* fix random_corpus_id for splicing mutator not considering disabled entries

* fmt

* update docs

* clippy

* fix corpus_btreemap not working

* fix clippy warnings

* fix python bindings

* typo in count_with_disabled implementations

* fix certain splicing mutators not considering disabled inputs

* rename count_with_disabled to count_all

* introduce count_disabled function

* update docs for count_all, count_disabled and count

* * introduce get_from_all and nth_from_all for corpus implementations so get() and nth() do not silently fetch disabled entries.
* remove boolean flag from random_corpus_id which allowed inclusion of disabled ids and make it into a new function random_corpus_id_with_disabled
* update docs

* remove boolean is_disabled flag from corpus::insert and make it into a separate function insert_disabled

* rename do_insert to _insert

* make get_from_all inline for cached and inmemory

* add missing functions implementation for PythonCorpus
prevent writing feedback when adding disabled testcases

* fix nth_from_all overfetching enabled corpus entries

* fix clippy & rename execute_with_res to execute_no_process

* refactor _insert for corpus_btreemap

* make LibfuzzerCorpus and ArtifactCorpus to accomodate disabled entries

* fix typo

* fix missing docs for map field

* fix clippy

* test

* (hopefully) fix CachedOnDiskCorpus using incorrect corpus when caching testcase

* fix typo in inmemory_ondisk leading to fetching disabled entry from enabled corpus

---------

Co-authored-by: aarnav <aarnav@srlabs.de>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-04-10 01:03:00 +02:00
David CARLIER
d3b3d5d462
bolts: add time ticks method for arm (#2032) 2024-04-10 01:01:27 +02:00
Dongjia "toka" Zhang
c4f076356f
Fix the OOM exit code from 137 to 9 (#2029)
* i don't know if there really exit 137

* change message

* not 137 but 9
2024-04-09 17:16:25 +02:00
Romain Malmain
33c6ea1ae4
Fix libafl_qemu python bindings and example (#2030) 2024-04-09 16:09:47 +02:00
Mrmaxmeier
f252469d86
CI: A few updates and fixes (#2023)
* CI: Don't run cargo clean in order to preserve build caches

* CI: run yaml formatter >:)

* CI: try sharing build caches between similar jobs

* CI: it looks like the `ucd-generate` tool is not used anymore

* CI: Sneak in merge queue support :)

* CI: split out `cargo fmt --check` job

* libafl_qemu injections: be more resilient about mapping paths
2024-04-09 12:55:07 +02:00
Dominik Maier
2cc560e727
Tiny clippy fix for qemu_launcher fuzzer (#2026) 2024-04-09 12:49:42 +02:00
Dominik Maier
ef25aef471
Fix docs for frida (see #2025) (#2027) 2024-04-09 12:48:24 +02:00