191 Commits

Author SHA1 Message Date
Dongjia Zhang
9d38fff662
Autodict forkserver (#525)
* Builder for ForkserverExecutor

* add

* clippy warnings

* comment

* stash

* tmp

* change

* revert

* use_shmem_feature field

* change the harness back

* wip

* wip

* revert

* works

* clippy

* Makefile fix

* doc

* clippy

* rename to program

* rename, fix, envs

* lifetime

* arg_input_file

* stash

* read autodict from forkserver

* works

* clippy & fmt

* fmt

* fix

* fix

* fmt

* better harness

* arg_input_file_std

* rename

* fix
2022-02-10 10:27:51 +01:00
Dongjia Zhang
9482433e54
Forkserver builder (#523)
* Builder for ForkserverExecutor

* add

* clippy warnings

* comment

* stash

* tmp

* change

* revert

* use_shmem_feature field

* change the harness back

* wip

* wip

* revert

* works

* clippy

* Makefile fix

* doc

* clippy

* rename to program

* rename, fix, envs

* lifetime

* arg_input_file

* bug fix

* arg_input_file

* builder()

* doc

* clippy & fmt

* clippy & fmt
2022-02-09 22:07:15 +01:00
Dominik Maier
98fbe83c15
Differential executor, diff feedback, stdio observers for command executor (#521)
* started diff fuzzer

* finished DifferentialExecutor

* adapt builder, more diff fuzz infra

* diff eq feedback

* stdout observer started:

* stdio observers

* stdio observers

* no_std, fixes

* no_std tests
2022-02-06 18:20:57 +01:00
Sagittarius-a
2bb60fb756
Fix documentation typos (#514)
* Fix typos in LibAFL doc comments

* Fix doc comment for ProgressReporter trait

* Remove unused comment

* Link ShMem by name in doc comment
2022-02-03 16:31:19 +01:00
Dongjia Zhang
3c4ec38d83
Win Fix (#513)
* win_fix

* fmt

* another fmt
2022-02-02 00:26:10 +01:00
Dominik Maier
6810e6085b
Builder for CommandExecutor & Tokens Refactoring (#508)
* builder for CommandExecutor

* tokens api cleanup, clippy

* fix doctest

* cleanup

* added testcase, remodelled

* command executor builder fix

* fix fuzzer(?)

* implemented From for configurator

* nits

* clippy

* unused

* autotokens

* cleanup

* nits

* Err instead of empty tokens

* fix tokens fn

* fix err

* more error fixing

* tokens remodelling

* typo

* recoverable fail on missing autotokens

* clippy, nostd

* asslice, into_iter, etc. for tokens

* adapt fuzzers

* iter

* fixes, clippy

* fix

* more clippy

* no_std

* more fix

* fixed typo

* cmd_executor builds again

* bring back ASAN stuff to Command Executor

* forkserver speedup

* no need to static

* back to earlier
2022-02-01 10:10:47 +01:00
Dongjia Zhang
c61fed6ca9
Use Unix timer_* API instead of setitimer (#510)
* fix linter errors for armv7 (docs)

* introduce HasOnCrashReset trait; use timer_* API instead of setitimer for unix TimeoutExecutor

* fixes: PR #469 annotations and CI issues

* reintroduce setitimer for apple as macOS does not feature the POSIX timer API

* more macos and windows CI fixes

* more macos and windows CI fixes cont.

* HasOnCrashReset -> HasPostRunReset

* remove drop impl for Windows TimeoutExecutor

* adjust target cfgs for timeout stuff (android also did not work)

* add call to inner post_run_reset

* remove HasPostRunReset in favor of making it a trait fn of Executor

* add post_run_reset's to CombinedExecutor

* clippy: addr_of! instead of raw pointer casts

* link librt in libafl_cc (required by timer_* API)

* minor fixes and cleanup

* remove unused import for targets other than linux

* fix win

* merge

* fix

Co-authored-by: pr0me <g33sus@gmail.com>
2022-02-01 04:48:03 +01:00
Dominik Maier
9dfc6aa404
CI and fixes for arm32 no_std build (#511)
* arm32 no_std fixes and clippy

* moved criterion to benches crate

* benches no longer live here
2022-02-01 00:57:58 +01:00
Youssef
e307dfb16f
Implement backtrace observers for crash dedupe (#379)
* create stacktrace observer

* create stacktrace feedback

* post-merge fixes

* address comments

* update Cargo.toml

* fix CI issue + dynamic naming

* duplicate baby_fizzer

* update stacktrace baby_fuzzer

* force unwinding tables

* ignore test dumps

* fix stacktrace baby_fuzzer logic

* upgrade Backtrace version

* trigger observers.post_exec in crash_handler

* implement NewHashFeedbackState and update logic

* digest symbols pointers

* cleanup

* minimal output

* fix backdated EventFirer generic param

* add baby_fuzzer example with a fork executor

* duplicate baby_fuzzer_stacktrace with forkexecutor

* backtrace collection implemented

* add c app fuzzer example with a fork executor

* group backtrace baby fuzzers

* added c code baby fuzzer with inprocess executor

* remove need for static COLLECT_BACKTRACE

* moved code to stacktrace.rs + fixed bug

* add comment

* add command executor fuzzer example

* post merge cleanup

* add missing doc

* address comment

* fix nit

* clean duplicate variable in timeout handler

* fix command executor bt collection

* clean code and use StdShMem

* cleanup

* add ObserverWithHashField + rename StacktraceObserver

* rename + refactor some code

* add CommandBacktraceObserver

* update command executor

* update baby fuzzers

* simplify BacktraceSharedMemoryWrapper

* use better names + static methods

* use std feature macro on BacktraceObserver + fix bug

* use Box in HashValueWrapper to minimize variants size diff

* use copy_from_slice

* std conditional backtrace collection

* fix std import

* fix comment

* add exit_kind to observer.post_exec

* added hash trait to Input

* collect backtrace in post_exec

* add crash handlers to InProcessForkExecutor

* fix panic message

* duplicate forkserver fuzzer example

minimal example

update

* proto bt collection working

* rename CommandBacktraceExecutor to ASANBacktraceExecutor

* refactor ASANBacktraceObserver

* support for forkserver working

* update fuzzer example

* less verbosity

* Post merge fixes

* implement hash for GeneralizedInput

* update forkserver example after merge

* clippy fixes

* fix inproc test

* fixes for cargo hack --feature-powerset

* fix baby_no_std

* implement Hash for NautilusInput

* update fork executor baby fuzzer

* fix doc

* implement Hash for PacketData

* fix windows build

* fix windows no_std

* fix backtrace baby fuzzers README

* add comments

* move setup_bt_panic to constructor

* pre/post child exec hooks in Observer

* setup_child_panic_hook

* fix ObserversOwnedMap on nightly

* add backtrace fuzzers to CI checks

* fix typo

* fix relative paths in test_all_fuzzers.sh

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-01-31 15:58:15 +01:00
Dongjia Zhang
5e1c0b96ea
Various Fixes (windows timeout race & frida options) (#482)
* race fix

* oops

* no backtrace
2022-01-20 01:32:04 +01:00
Dominik Maier
77e5965e97
Add AsSlice, AsMutSlice traits, refactor MapObservers to be iterable, and have associated types (#477)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic

* refacctor map observers

* iterator for map observers

* removed unused ownedptr, added asslice trait to most functions

* make map entry type an associated type

* fix fuzzers

* fix docs

* typo fix

* fix windows, add try_from_slice to shmid

* missing import

* fix fuzzbench

* cleanup

* fmt

* more asslice

* fmt

* added doc link about token-level fuzzing

* cods
2022-01-19 00:02:33 +01:00
Dominik Maier
4f6f76e857
Streamline ShMem API (#472)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic
2022-01-17 18:28:26 +01:00
buherator
5ac3cd6b5a
Optional signal value for kill on timeouts in TimeoutForkserverExecutor (#461)
* Optional signal value to kill forked processes on timeout

* Cargo format

* Properly initialize TimeoutForkserverExecutor

* Added with_signal constructor

* Removed duplicate code
2022-01-09 14:31:14 +01:00
Evan Richter
250ec8d1e0
Reduce generics for various Has* traits (#456)
Specifically for Has{Rand,Corpus,Solutions,FeedbackStates}

The Has* family of traits offer getters and get-mut-ers. The previous
implementation had a fully generic return type:

    trait HasX<X: TraitX> {
        get_x(&self) -> &Self::X;
        get_mut_x(&mut self) -> &mut Self::X;
    }

meaning a single type could implement both `HasRand<Romu>` and
`HasRand<XorShift>`. The advantage of having multiple implementations is
not clear at this time, so it vastly simplifies the trait (and its
impls) to bring the return type in the body as an associated type:

    trait HasX {
        type X: TraitX;
        get_x(&self) -> &Self::X;
        get_mut_x(&mut self) -> &mut Self::X;
    }

This comes with the limitation that any type that impls these traits can
only do so once, choosing only one associated type.

* HasRand's only generic parameter (Rand) is now an associated type
* HasCorpus and HasSolutions are now only generic over the Input type
  they store
* HasFeedbackStates generic parameter now associated type
2022-01-06 10:41:02 +01:00
Dongjia Zhang
674005fa61
Reorder type parameters in the correct order (#449)
* alphabetical order

* revert

* revert

* fix
2022-01-04 00:20:29 +01:00
Dominik Maier
af3d321213
Derive debug for all structs in LibAFL (#442)
* documentation, warnings

* fixed docs

* docs

* no_std

* test

* windows

* nautilus docs

* more fixes

* more docs

* nits

* windows clippy

* docs, windows

* nits

* debug all the things

* derive debug for all core library components

* Docu for libafl_targets

* nits

* reordered generics

* add docs to frida, debug

* nits

* fixes

* more docu for frida, nits

* more docu

* more docu

* Sugar docs

* debug for qemu

* more debug

* import debug

* fmt

* debug

* anyap_debug feature no longer needed

* tidy up unused fn

* indicate if we left out values for struct debug

* implement Debug for sugar

* debug allthethings

* ci
2022-01-03 00:47:17 +01:00
Dominik Maier
cb3662da54
Enable errors for missing docs, add documentation (#440)
* documentation, warnings

* fixed docs

* docs

* no_std

* test

* windows

* nautilus docs

* more fixes

* more docs

* nits

* windows clippy

* docs, windows

* nits
2022-01-01 19:51:27 +01:00
Dongjia Zhang
d697554810
Other/User defined WIndows Exceptions (#402)
* other exceptions

* add

* 46th

* fix

* fmt
2021-12-21 19:18:58 +01:00
Dongjia Zhang
79f9bcd3e0
Use AddVectoredExceptionHandler to register exception handlers (#403)
* add

* unix fix

* unsafe positions

* another unsafe!

* ignore

* ignore

* make changes back

* fix

* fix

* fmt

* exception fix

* fix

* bug fix

* fmt

* fix things messed up during merge

* stack overflow fix

* fix

* fix

* fix

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-12-16 11:15:24 +01:00
Dominik Maier
217a7dee1d
Use Structopt instead of yaml for example fuzzers, introduce Cores API (#420)
* reworked generic_inmemory to structopt

* moved core parsing to a struct

* added Cores

* added structopt to libpng_ctx

* improved libafl, added structopt to libpng launcher

* fix deexit ub

* move more to structopt

* improve llvm-config detection

* move construct_automata to structopt

* clippy, fixes, ...

* no_std

* clippy

* frida core parsing

* fixed no-fork cores

* updated clap

* added missing import

* missing borrow

* reworked frida to structopt

* fixed build

* using Cores api for atheris

Co-authored-by: Dominik Maier <d.maier@avm.de>
2021-12-15 03:58:35 +01:00
Andrea Fioraldi
b4c2551544
Debug output for forkserver (#413)
* usability fixes for forkserver

* don't call target_bytes twice in TimeoutForkserverExecutor

* don't call target_bytes twice in ForkserverExecutor
2021-12-10 14:52:23 +01:00
Dongjia Zhang
3fbe1be189
Fix timeout value type for Windows (#414) 2021-12-09 20:08:44 +01:00
Dongjia Zhang
fc0881194d
Windows timeout fix with critical sections (#391)
* add

* unix fix

* unsafe positions

* another unsafe!

* ignore

* ignore

* make changes back

* fix

* fix

* fmt

* bug fix

* fmt

* compiler fence

* import

* typo

* add another critical section

* fix

* fix

* exclude windows book test

* typo

* fence

* why

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-12-09 13:55:20 +01:00
van Hauser
4a23489acb
Implement unstable edge detection+ignore in calibration stage (#398)
* step 1 for unstable calibration

* fmt

* fixed build

* done unstable implementation

* clippy

* finishing touches for unstable

* no_std

* fmt

* event mgr stablity

* fixed stability value

* displaying

* no_std

* fixed critical whitespace

* send msg only after calibration

* clippy

* Added log to mgr

* moved stability to state

* fix introspection

* space

* fixed docs

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-12-08 09:54:47 +01:00
Dominik Maier
83583a867f
QEMU target arch selector via feature flag (#405)
* QEMU target arch selector via feature flag

* fix ci

* fixing ci some mmore

* more ci fixes, defaulting to x86_64 always

* more ci

* i368 -> i386 typo fix

* revert forkserver changes

* trying to fix clippy

* docs

* fixed warnings

* more clippy action

* qemu example arch

* bring back deprecated function I don't know how to replace

* get rid of deprecated feature again'

* builds?i
2021-12-06 20:06:47 +01:00
Lukas Seidel
0d0fe9eaca
Add fn for TimeoutExecutor to change desired timeout (#408) 2021-12-06 17:44:24 +01:00
Andrea Fioraldi
2a780652e1 Fix HasTimeoutHandler 2021-11-18 09:42:35 +01:00
Dominik Maier
01bb599759 fixed ms 2021-11-17 20:18:50 +01:00
Andrea Fioraldi
5605f233fc
InProcessHandlers (#387)
* InProcessHandlers

* clippy
2021-11-17 18:32:26 +01:00
Dominik Maier
ba969108e3
Push stage trait (#380)
* rpush mutational trait

* tiny changes

* started PushStageAdapter

* fmt

* refactoring

* fix docs

* no_std

* formatted more
2021-11-17 12:51:14 +01:00
Dominik Maier
62afed61e2
Renamed Stats to Monitors (#373)
* renamed stats to monitors

* added depreciation notices

* resorted generics alphaabetically

* added monitors

* fmt fuzzers

* added depreciation note for usermonitor

* fmt all fuzzers script

* more fmt

* renamed some monitor things back to stats

* fixed rename
2021-11-12 11:01:08 +01:00
Dominik Maier
3e85cf22de
Mutational Push Stage (#356)
* initial commit for push stage

* cleanup, no_std, clippy

* clippy

* fuzzes

* readme

* fmt
2021-11-11 01:49:38 +01:00
s1341
dd0b5fa74f
Add minibsod (#362)
* Add minibsod

* fmt'

* clippy

* nostd/mac fixes

* windows fix

* woops. Mac fixes

* Get rid of unneccesary sleep

* Fix missing unsafe

* clippy fixes

* make ucontext,siginfo not a reference

* fmt

* fix _context

* Add stubs for non-apple, non-linux, non-android; add a todo

* Fmt

* macos x64, testcase, cleanup

* no_std

* added fault address to minibsod for apple x64

* added err, hexlified values (as per mac panic)

* informing user about lack of registers

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-11-07 15:32:43 +01:00
Dominik Maier
ea820a1694
Fork feature flag to disable fork in Launcher (#351)
* Fork feature flag to disable fork in launcher

* FnOnce instead of FnMut

* cleanup

* fix no_std

* doc/warning fix

* useless clippy fix
2021-11-05 17:26:49 +01:00
Jordan McLeod
3d436b7519
Upgrade to Rust 2021 Edition (#340)
* Enable missing const_xxh3 feature

* Move to Rust 2021 Edition

* Fix clippy complaints

* Remove imports made unecessary in 2021 edition
2021-11-04 09:59:49 +01:00
David CARLIER
939784d512
dumping process address maps on netbsd (#316) 2021-10-06 00:18:40 +02:00
Andrea Fioraldi
90928d3d97
Qemu Helpers and basic snapshotting (#310)
* store executor_ptr

* QemuHelpers

* working hooks and snapshot helper

* walk only the list of dirty pages on restore()

* mem hooks for snpashot

* brk snapshot

* snapshot method

* macos shit

* sugar and clippy
2021-10-01 12:17:28 +02:00
Dominik Maier
5a722994ac
Still fixing CI (#305)
* clippy

* clippy on windows

* clippy fixes on windows

* clippy, fmt

* fixed testcases for windows

* fixing workspace.yml

* testcase no longer fails without clang

* fix github dependencies to specific revisions

* fix qemu without python

* cleanup HookResult
2021-09-29 17:58:09 +02:00
Andrea Fioraldi
c6f7c3b3a8
Qemu new syscall hook and more python API (#306)
* new syscall hook

* expose more qemu to pylibafl

* hook syscalls from python

* update python example

* clippy

* clippy
2021-09-29 16:36:40 +02:00
Andrea Fioraldi
05aeb677cf
Fix default UBSan options and avoid timeouts in crash handler (#304)
* exit

* ignore timeouts outside the targets

* do not store timeouts

* block sigalarm in handlers

* __ubsan_default_options
2021-09-29 09:28:55 +02:00
Lukas Seidel
f0d5c2f708
armv7 support: add ucontext struct definition (#297)
* add uncontext and mcontext struct defs for armv7

* formatting

* merge import lines
2021-09-14 13:58:03 +02:00
Dominik Maier
55c4b0c778
added write_file_atomic against ondisk corpus races (#294)
* fix ondisk corpus race condition

* move metadata name to be a dotfile

* note ExitKind for crashes and timeouts in inprocess executor

* potential fix for windows

* added write_file_atomic

* no_std fixes

* no_std testcase fix

* typo fix, windows

* clippy

* more no_std testing
2021-09-13 15:38:28 +02:00
David CARLIER
774cfb685e
openbsd port (#279)
with libc update, ucontext data is finally available on this platform too.
2021-08-27 11:41:06 +02:00
Dominik Maier
5caeb46b67
renamed target_os macos to target_vendor apple (#273)
* renamed target_os macos to target_vendor apple

* fix yaml
2021-08-23 09:45:25 +02:00
David CARLIER
8f03d20200
process crash handler, dump registers on apple arm64 (#271) 2021-08-21 14:47:55 +02:00
Toka
b6d22a2fbb
Drop for TimeoutExecutor on win10 (#269)
* small fixes

* update .gitignore
2021-08-19 09:11:16 +02:00
Toka
688182fd1e
Timeout for Inprocess Executor on Windows (#267)
* start working on windows timeout

* salvage Input in timeout handler

* this time inproc_timeout_handler (need clean up later)

* cleaup

* more in inproc_timeout_handler

* fix for linux build

* more fixes for unix, fmt

* revert timeoutexecutor api

* revert baby_fuzzer/src/main.rs

* various fixes

* no unsafe

* remove timer in crash_handler
2021-08-18 09:11:34 +02:00
Toka
d7ec395010
Forkserver Shared Memory Testcase (#265)
* working on shmem testcase fuzzing

* fmt & clippy

* write_to_testcase

* write input size

* max os fixes

* RcShMemProvider?

* ServedShMemProvider?

* revert changes

* RcShMem<ServedShMemProvider<MmapShMemProvider>>?

* ShMem change for android? (not tested at all)

* harness

* shmem testcase fuzzing for timeoutforkserver

* update harness

* remove .o

* pselect instead of select

* clippy
2021-08-17 01:44:17 +02:00
Toka
511237ce9e
Windows dependency update (#261)
* win018

* fuzzer fmt

* reorder

* comment

* does
2021-08-14 23:04:13 +02:00
Dominik Maier
bb21ab7a63
Autolaunch ShMemService, add MacOS fuzzers to CI, various fixes (#246)
* starting to fix macos linker bugs

* mdetailed error prints

* start shmem service manually

* not a doc comment

* Some fixes

* only send exit msg to shmemservice when start was successful

* incorporated shmem service into provider

* removed unused imports

* trying to fix fuzzers

* fixed build

* check if join_handle is_some

* more debug prints

* fixed shmem service autolaunch

* fixed macos linker

* ignoring broken libpng testcase on macos for now (see #252)

* fmt

* try to fix missing llvm_config (see #253)

* empty issue template added

* Mmanually look for llvm-config on MacOS

* fixing CI

* fixed docs

* ignoring libmozjpg for CI, see #254
2021-08-09 01:10:29 +02:00