1079 Commits

Author SHA1 Message Date
Dominik Maier
e8838ebebe
Safer EoP handling (#1128) 2023-03-08 00:33:55 +01:00
Andrea Fioraldi
20c32316eb
Define custom collectors for QemuCallTracerHelper (#1099)
* Define custom collectors for QemuCallTracerHelper and create OnCrashBacktraceCollector

* fmt

* clippy
2023-03-07 13:16:51 +01:00
Andrea Fioraldi
3ffec79a17
Increase LLMP clients timeout to 5 min (#1126)
* LLMP client access fast path

* Increase LLMP client timeoit to 5min

---------

Co-authored-by: Your Name <you@example.com>
2023-03-07 11:47:45 +01:00
Elia Geretto
4f7b59aca4
Forward on_evaluation callback in MinimizerScheduler (#1122) 2023-03-05 22:00:32 +09:00
Langston Barrett
35e5b87188
Cargo feature to avoid regex dependency (#1102)
regex is a large crate, and is only used in a few specific spots. Users should
have the ability to avoid this transitive dependency if not using the features
in question.
2023-03-03 16:00:49 +01:00
Dongjia "toka" Zhang
5b4ae61cdd
SimpleLogger API improvements, printing to stdout, timestamps (#1109)
* log

* fix

* a

* rev

* remove

* 2 logger

* cfg std

* more

* more

* cf

* no_std

* features

* optional

* rename

* current_time()
2023-03-02 14:07:46 +01:00
van Hauser
2a3f1d68f5
fix weighting from hits (#1120) 2023-03-02 14:05:34 +01:00
Dongjia "toka" Zhang
2a926f6546
Fix forkserver regression in LTO mode (#1114)
* regression

* fix

* chg

* Comment

* fmt
2023-03-02 10:17:20 +09:00
Andrea Fioraldi
452ca7a672
Implement EcoFuzz (#1115)
* Implement EcoFuzz

* clippy

* fix
2023-03-01 13:21:43 +01:00
Dominik Maier
31357aa7e2
Track parent testcase id, tuneable stage probabilistic settings (#1081)
* Added local event handlers

* clippy

* move tuned mutator to pow2

* Tunable updates

* parent ids

* no_std, etc

* windows

* remove local event manager handler

* maybe fix win

* win:

* win docs

* docs

* ASAN -> ASan
2023-02-28 16:36:04 +01:00
Dongjia "toka" Zhang
3e7322e395
Remove unnecessary check in calibration stage (#1111)
* save

* fix
2023-02-28 15:41:17 +01:00
Dominik Maier
fbe8cce1b8
Real OnDiskCorpus (#1096)
* Real OnDiskCorpus

* clippy

* python

* docs

* clippy

* docs

* move to reuse cachedinmem corpus

* fmt
2023-02-28 15:41:05 +01:00
Andrea Fioraldi
d36296c654
Fix llmp CliendId search (#1112) 2023-02-28 15:20:24 +01:00
Andrea Fioraldi
dc800f0814
on_evaluation Scheduler method (#1106)
* add on evaluation hook in schedulers

* on_evaluation for WeightedScheduler

* fix PowerQueueScheduler

* fix fuzzers

* upd qemu

* tests

* upd
2023-02-28 11:33:26 +01:00
Dongjia "toka" Zhang
59bf118a5a
Clippy (#1105) 2023-02-28 01:08:19 +09:00
Dongjia "toka" Zhang
c0f229ec23
Add UsesObserver to allow more generic MapFeedback (#1104)
* UsesObserver

* a

* more

* don't need these
2023-02-26 08:35:45 +01:00
clesmian
cf02553ea7
Cleanup forkserver exec builder (#1094)
* Don't use magic string but string constant

* Don't allow to specify multiple input files with different names

* Ensure that the file name for the current test case is unique for every fuzzer currently running

* Add note advising users to choose distinct names for the input file

* Move builder functions to more generic implementation to allow parse_afl_cmdline rewrite

* Rewrite parse_afl_cmdline to reduce code duplication

* Add remark to documentation regarding the program path

* Change behavior to allow the usage of actual AFL command lines, hopefully without breaking existing code

* Rustfmt

* Move generation of unique filename to fs

* Ensure default input filename for command executor is unique per fuzzing process

* Pass the input to the target via stdin, when no input file is specified

Previous solution of passing it via a standard file is useless, as the target does not know to read said file

* Rustfmt

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-02-26 02:00:28 +01:00
Langston Barrett
3dbea91a63
Use Iterators as Generator (#1101)
Also, remove seemingly-unused generate_dummy method to enable this instance.

Also, add an adapter that creates an Iterator from a Generator.
2023-02-26 01:59:56 +01:00
Dominik Maier
b3020d7296
Fix CoreId for Frida, FreeBSD (#1100) 2023-02-25 00:16:37 +01:00
Dominik Maier
672d25e5ac
LLMP Client timeouts, Exit broker when last client exits (#1057)
* Moving type definitions to transparent structs

* function to notify other side of exit

* docs

* Exmaple support windows now

* timeout fix

* Exiting after the last client quit

* inform about quits

* clippy

* clippy

* clean exits

* fix

* more unsafe

* fixes

* Move ClientId

* fix no_std

* Fix prometheus

* introduce Cores.trim()

* add always_track metadata

* docu

* add AlwaysUniqueMapFeedback

* rename to always_interesting

* return CoreId for Launcher

* CoreId as transparent tuple struct

* fix graceful exits for launcher

* Broker exits after launcher

* clippy

* Fix llmp eop race, introduce llmp shmem cache

* initialize cached page, clippy

* fix llmp_debug strings

* add error handling

* nicer error output

* More error handling convenience

* clippy

* fix macos example

* nits

* trying to add a logger

* no_std

* inline logger enabled

* fix windows, non-fork

* macos

* no_std docs

* clippy

* use ? instead of unwraps in example

* more logging

* docs
2023-02-24 11:50:42 +01:00
Dominik Maier
92842c8b04
Fix LLMP eop race, introduce LLMP ShMem cache (#1091)
* Fix llmp eop race, introduce llmp shmem cache

* initialize cached page, clippy

* fix llmp_debug strings

* add error handling

* nicer error output

* More error handling convenience

* clippy

* fix macos example

* nits

* trying to add a logger

* no_std

* inline logger enabled

* clippy
2023-02-24 10:28:21 +01:00
Mrmaxmeier
ff4e2f4192
Fix max input size for {CrossOverInsert,BytesInsertCopy}Mutator (#1097) 2023-02-24 00:07:54 +01:00
R. Elliott Childre
64a57ad3e3
Move bytecount to dev-dependencies (#1090)
It is only used in test code
2023-02-23 13:52:43 +01:00
Dongjia "toka" Zhang
9e88e5734e
Timeout executor cfg fix (#1088)
* fix

* fix

* fix from mac

* fix

* fix

* fix

* ?

* fix
2023-02-22 10:20:11 +09:00
Addison Crump
b7296db406
Fix exits which may cause double-free corruption (#1086) 2023-02-21 07:34:16 +09:00
Addison Crump
bdac876dd4
Mutator sampling probability fixes (#1030)
* fixes for standard mutations

* more mutation updates for sampling probability, tests

* slight doc fix

* clippy gripe

* clippy fixes

---------

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2023-02-16 17:29:57 +01:00
Michael Rodler
46b75747ef
Make sure input was loaded to avoid panic on unwrap in MutatedTransform (#1077)
* make sure input was loaded to avoid panic on unwrap
fixes issue #1059

* avoid unnecessary clone, avoid unnecessary branching

---------

Co-authored-by: Michael Rodler <m@mrodler.eu>
Co-authored-by: Addison Crump <addison.crump@cispa.de>
2023-02-16 17:29:33 +01:00
Addison Crump
f454d17482
CMin: don't add to map if it's the initial value (uninteresting) (#1078) 2023-02-16 17:28:49 +01:00
Dongjia "toka" Zhang
8bffd28b4c
fix (#1076) 2023-02-16 10:29:24 +09:00
Dongjia "toka" Zhang
e7ef6ae8b7
Forkserver: 1. Add mem barrier 2. Don't send the initial 4 bytes message when it uses dynamic map option only (#1073)
* fix

* Real fix

* a
2023-02-16 09:42:28 +09:00
Addison Crump
26aace6073
Fix grimoire when used with on_replace/on_remove (#1075) 2023-02-16 00:14:26 +01:00
Addison Crump
e42cd9c12f
Fixes for on_replace/on_remove and related for StdFuzzer and MapFeedback (#1067)
* scheduler replace fixes

* oops, no-std

* add

* changes on the fuzzers

* move map feedback history updates to append_metadata

* fixes for python bindings

* learn to clippy

* fix for fuzzer add_input

* clippy fixes for frida

* additional powersched differences

* corrections for bitmap_size

* off-by-one

* I live in a prison of my own creation and clippy is the warden

* clear the novelties map for the situation where is_interesting is invoked, but not append_metadata

---------

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2023-02-15 17:04:18 +01:00
radl97
e61ac10656
Fix StdErrObserver not implementing needed traits (#1072) 2023-02-15 12:39:23 +01:00
Vincent
71d367af30
TimeObserver: Use Instant::now instead of Duration (#1064)
* Use Instant::now instead of duration

* Use Some

* add custom serde for Instant

* fix linter

* only enable TimeFeedback when std flag is enabled

* fix typo

* fix linter std

* cargo fmt

* allow clippy::trivially_copy_pass_by_ref on custom serde serialize function

* allow TimeObserver and Timefeedback for no_std

* cargo fmt

---------

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2023-02-15 12:38:25 +01:00
David CARLIER
fb0d3b07ea
MiniBSoD: solaris on amd64 implementations (#1068) 2023-02-15 12:27:45 +01:00
Fabian Freyer
53dba5f49d
Use the log facade instead of println (#1060)
* switched a couple of println / dbg statements to use log crate
* Use pyo3-log for logging in python bindings
2023-02-14 10:01:51 +01:00
Dongjia "toka" Zhang
bd2de16b4e
Colorization stage (#1039)
* type_replace

* separate

* more

* heap

* comment

* f

* fix

* clp

* need rev

* comment

* ColorizationTracingStage

* get_raw_map_hash_run

* process_execution

* metadat

* unused TE

* resolve type errors

* remove colorizationtracingstage

* Finally compiles

* clp

* fmt

* a few debug println

* revert

* fix
2023-02-13 15:02:19 +09:00
Dongjia "toka" Zhang
4d78878c02
Send stability in calibration stage & FridaInstrumentationHelper retunrs Result<Self, Error> (#1056)
* fix

* fix

* clippy
2023-02-13 05:35:09 +09:00
Dongjia "toka" Zhang
b7a0b823c6
Fix frida_gdiplus (#1045)
* fix

* I don't like prelude

* clp

* cargo make test

* poc

* one to_vec()

* fix?

* del

* fix
2023-02-08 00:20:38 +09:00
Andrea Fioraldi
0173d722c6
Fix typo in directory visiting (#1050) 2023-02-07 10:47:36 +01:00
Dongjia "toka" Zhang
30e296968b
Rename LLMP Timeout message, increase timeout (#1048)
* rename

* Update llmp.rs

* Print every 30 seconds only

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2023-02-06 16:42:14 +01:00
Langston Barrett
4e15be182e
Increase default initial capacity of NewHashFeedback (#1049) 2023-02-06 15:50:26 +01:00
R. Elliott Childre
4d5a759955
Update deps for libafl (#1042)
Reduces total number of packages from 577 to 571 on building with:
`cargo +nightly build --workspace --all-features`

* ahash 0.7 -> 0.8
  * Move `AHasher::new_with_keys` to `RandomState::with_seeds` given the
    recommendation from: aHash maintainer:
    https://github.com/tkaitchuck/aHash/issues/132#issuecomment-1288207069

* bindgen: 0.61 -> 0.63

* c2rust-bitfields: 0.3 -> 0.17

* criterion: 0.3 -> 0.4

* crossterm: 0.25 -> 0.26

* dynasmrt: 1.2 -> 2

* goblin: 0.5.3 -> 0.6

* hashbrown: 0.12 -> 0.13

* nix: 0.25 -> 0.26
  * The `addr` arg of `mmap` is now of type `Option<NonZeroUsize>`
  * The `length` arg of `mmap` is now of type `NonZeroUsize`
  * Requires updating implementers to update `nix` as well

* prometheus-client: 0.18.0 -> 0.19
  * Do not box metrics
  * Gauges (a majority of the LibAFL metrics) are now i64 types so there
    is a small chance of overflow, with the u64 values that LibAFL
    tracks, but unlikely to be problematic.
 * Keep `exec_rate` as a floating point value

* serial_test: 0.8 -> 1

* typed-builder: 0.10.0 -> 0.12

* windows: 0.42.0 -> 0.44

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-02-06 12:24:42 +01:00
Langston Barrett
e75f65080e
libafl: with_capacity method for NewHashFeedback (#1034)
Most of the time, fuzzing campaigns are reasonably long. Accordingly, when
using `NewHashFeedback`, you might have good reason to believe that you'll
find thousands (or more) different observations with different hashes. When
the `HashSet` outgrows its capacity, it can cause reallocation, which is slow.
See the following link for more details:

https://doc.rust-lang.org/std/vec/struct.Vec.html#capacity-and-reallocation
2023-02-06 01:44:57 +01:00
Dominik Maier
48caffb802
Allow to load a list of files (#1044) 2023-02-05 21:22:58 +01:00
Andrea Fioraldi
eaf5ff9de0
Restart loading initial inputs even after a crash/timeout (#1040)
* Track initial inputs loading

* libfuzzer libpng

* fuzzbench

* fix no_std

* fix no_std

* clippy

* fuzzers
2023-02-03 11:56:47 +01:00
Dominik Maier
cc53da85fb
Remove {update,clear}_hash from ObserverWithHashField, add hasher (extending #1019) (#1028)
* libafl: Remove `{update,clear}_hash` from `ObserverWithHashField`

These methods aren't used by `NewHashFeedback`, so there's no compelling reason
to keep them in the interface. They preclude implementations of
`ObserverWithHashField` that calculcate a hash on-the-fly from a value. For
example, my use-case is to store the stdout of a process, and use
`NewHashFeedback` to only collect inputs that result in new messages on stdout.

Both of these methods are pretty suspicious to begin with - why should other
code be able to update the internal state of the observer? What are the
semantics of `update_hash`? If there are compelling reasons to keep these
methods, let's clarify their intent in the documentation.

* libafl: Return hash by value from `ObserverWithHashField`

This allows implementors of this trait to not store the hash, but rather to
compute it on-the-fly. Since `Option<u64>` is `Copy` (and quite small), and
this method is called once per execution of the target program, this is likely
to have negligible performance impact.

* libafl: Implement `ObserverWithHashField` for `ValueObserver`

This demonstrates the utility of the previous two commits. Now, `ValueObserver`
can be used with `NewHashFeedback`.

* Clippy, move to ahasher

* Oops :)

---------

Co-authored-by: Langston Barrett <langston.barrett@gmail.com>
2023-01-31 10:45:42 +01:00
Andrea Fioraldi
fdf579bcd5
Bump to 0.9.0 (#946)
* bump to 0.9.0

* fix libafl_tinyinst

* fix

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-01-31 10:42:01 +01:00
Dominik Maier
33ddce2cea
Introduce MutatorId, Tuneable fixes (#1022)
* Add simpler APIs for TunableStage

* Make API usable

* Add TunableScheduledMutator APIs

* Introduce MutatorId

* More API

* Cleanup

* add sampling-based mutation scheduling

* reduce precision for sampling

* clippy
2023-01-30 18:04:42 +01:00
Dongjia "toka" Zhang
b927fc9b06
Add filename_path to MmapShMemProvider (#1014)
* change how it is named

* Add more comments

* more

* macOS 32bytes onlyu

* chg

* comment, fix
2023-01-26 00:53:37 +01:00