37 Commits

Author SHA1 Message Date
Dominik Maier
663a33168e
Associated types for Corpus, State (#767)
* Associated types for Corpus, State

* cleanup

* fix no_std

* drop unused clauses

* Corpus

* cleanup

* adding things

* fixed fuzzer

* remove phantom data

* python

* progress?

* more more

* oof

* wow it builds?

* python fixes, tests

* fix python fun

* black fmt for python

* clippy, added Nop things

* fixes

* fix merge

* make it compile (#836)

* doc-test fixes, prelude-b-gone for cargo-hack compat

* fixes for windows, concolic

* really fix windows, maybe

* imagine using windows

* ...

* elide I generic when used with S: State

* Elide many, many generics, but at what cost?

* progress on push

* Constraint HasCorpus, HasSolutions at trait definition

* remove unused feature

* remove unstable usage since we constrained HasCorpus at definition

* compiled, but still no type inference for MaxMapFeedback

* cleanup inprocess

* resolve some std conflicts

* simplify map

* undo unnecessary cfg specification

* fix breaking test case for CI on no-std

* fix concolic build failures

* fix macos build

* fixes for windows build

* timeout fixes for windows build

* fix pybindings issues

* fixup qemu

* fix outstanding local build issues

* maybe fix windows inprocess

* doc fixes

* unbridled fury

* de-associate State from Feedback, replace with generic as AT inference is not sufficient to derive specialisation for MapFeedback

* merge update

* refactor + speed up fuzzer builds by sharing build work

* cleanup lingering compiler errors

* lol missed one

* revert QEMU-Nyx change, not sure how I did that

* move HasInput to inputs

* HasInput => KnowsInput

* update bounds to enforce via associated types

* disentangle observers with fuzzer

* revert --target; update some fuzzers to match new API

* resolve outstanding fuzzer build blockers (that I can run on my system)

* fixes for non-linux unixes

* fix for windows

* Knows => Uses, final fixes for windows

* <guttural screaming>

* fixes for concolic

* loosen bound for frida executor so windows builds correctly

* cleanup generics for eventmanager/eventprocessor to drop observers requirement

* improve inference over fuzz_one and friends

* update migration notes

* fixes for python bindings

* fixes for generic counts in event managers

* finish migration notes

* post-merge fix

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2022-10-24 03:22:26 +02:00
Dominik Maier
e8b3d33bf4
Update dependencies, removed unused deps, CI fixes (#839)
* update clap, remove unused deps

* update grammartek

* update pyo3

* update pyo3

* undid clap update

* not changing nyx

* updated deps

* Update more deps, fixes

* not needed clippy

* fix windows

* try to enable deprecated pyproto for pyo3

* unused

* moving some things to clap4 after all

* initial move to clap 4

* fix clap

* more clap4, removed accidental file

* fixes, fmt

* fix

* all fix no play

* fix
2022-10-18 20:36:43 +02:00
Dongjia "toka" Zhang
dee3bc4492
Fix windows timeout 2022-10-17 10:00:59 +09:00
syheliel
2504b6dae3
Add rustfmt.toml (#722)
* add `rustfmt.toml`

* format fix

Co-authored-by: syheliel <syheliel@gmail.com>
2022-08-12 02:28:32 +02:00
Dominik Maier
7d2892a42f
Fix Windows import (#642) 2022-05-22 13:07:45 +02:00
Dominik Maier
c404825fb8
More clippy (#641)
* Even more libafl_frida clippy

* Eq

* addr_of_mut cleanup

* fmt
2022-05-22 13:01:55 +02:00
Andrea Fioraldi
e8f5949aec
Fix linking with -z defs (#601)
* Always link no-link-rt when not linking a fuzzer

* Handle dynamic

* fuzzbench

* Handle -z defs

* fix

* clippy

* clippy

* windowa

* fix
2022-04-08 18:06:27 +02:00
Sagittarius-a
2bb60fb756
Fix documentation typos (#514)
* Fix typos in LibAFL doc comments

* Fix doc comment for ProgressReporter trait

* Remove unused comment

* Link ShMem by name in doc comment
2022-02-03 16:31:19 +01:00
Dongjia Zhang
c61fed6ca9
Use Unix timer_* API instead of setitimer (#510)
* fix linter errors for armv7 (docs)

* introduce HasOnCrashReset trait; use timer_* API instead of setitimer for unix TimeoutExecutor

* fixes: PR #469 annotations and CI issues

* reintroduce setitimer for apple as macOS does not feature the POSIX timer API

* more macos and windows CI fixes

* more macos and windows CI fixes cont.

* HasOnCrashReset -> HasPostRunReset

* remove drop impl for Windows TimeoutExecutor

* adjust target cfgs for timeout stuff (android also did not work)

* add call to inner post_run_reset

* remove HasPostRunReset in favor of making it a trait fn of Executor

* add post_run_reset's to CombinedExecutor

* clippy: addr_of! instead of raw pointer casts

* link librt in libafl_cc (required by timer_* API)

* minor fixes and cleanup

* remove unused import for targets other than linux

* fix win

* merge

* fix

Co-authored-by: pr0me <g33sus@gmail.com>
2022-02-01 04:48:03 +01:00
Dominik Maier
af3d321213
Derive debug for all structs in LibAFL (#442)
* documentation, warnings

* fixed docs

* docs

* no_std

* test

* windows

* nautilus docs

* more fixes

* more docs

* nits

* windows clippy

* docs, windows

* nits

* debug all the things

* derive debug for all core library components

* Docu for libafl_targets

* nits

* reordered generics

* add docs to frida, debug

* nits

* fixes

* more docu for frida, nits

* more docu

* more docu

* Sugar docs

* debug for qemu

* more debug

* import debug

* fmt

* debug

* anyap_debug feature no longer needed

* tidy up unused fn

* indicate if we left out values for struct debug

* implement Debug for sugar

* debug allthethings

* ci
2022-01-03 00:47:17 +01:00
Dominik Maier
cb3662da54
Enable errors for missing docs, add documentation (#440)
* documentation, warnings

* fixed docs

* docs

* no_std

* test

* windows

* nautilus docs

* more fixes

* more docs

* nits

* windows clippy

* docs, windows

* nits
2022-01-01 19:51:27 +01:00
Dominik Maier
217a7dee1d
Use Structopt instead of yaml for example fuzzers, introduce Cores API (#420)
* reworked generic_inmemory to structopt

* moved core parsing to a struct

* added Cores

* added structopt to libpng_ctx

* improved libafl, added structopt to libpng launcher

* fix deexit ub

* move more to structopt

* improve llvm-config detection

* move construct_automata to structopt

* clippy, fixes, ...

* no_std

* clippy

* frida core parsing

* fixed no-fork cores

* updated clap

* added missing import

* missing borrow

* reworked frida to structopt

* fixed build

* using Cores api for atheris

Co-authored-by: Dominik Maier <d.maier@avm.de>
2021-12-15 03:58:35 +01:00
Dongjia Zhang
3fbe1be189
Fix timeout value type for Windows (#414) 2021-12-09 20:08:44 +01:00
Dongjia Zhang
fc0881194d
Windows timeout fix with critical sections (#391)
* add

* unix fix

* unsafe positions

* another unsafe!

* ignore

* ignore

* make changes back

* fix

* fix

* fmt

* bug fix

* fmt

* compiler fence

* import

* typo

* add another critical section

* fix

* fix

* exclude windows book test

* typo

* fence

* why

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-12-09 13:55:20 +01:00
Lukas Seidel
0d0fe9eaca
Add fn for TimeoutExecutor to change desired timeout (#408) 2021-12-06 17:44:24 +01:00
Dominik Maier
5a722994ac
Still fixing CI (#305)
* clippy

* clippy on windows

* clippy fixes on windows

* clippy, fmt

* fixed testcases for windows

* fixing workspace.yml

* testcase no longer fails without clang

* fix github dependencies to specific revisions

* fix qemu without python

* cleanup HookResult
2021-09-29 17:58:09 +02:00
Toka
b6d22a2fbb
Drop for TimeoutExecutor on win10 (#269)
* small fixes

* update .gitignore
2021-08-19 09:11:16 +02:00
Toka
688182fd1e
Timeout for Inprocess Executor on Windows (#267)
* start working on windows timeout

* salvage Input in timeout handler

* this time inproc_timeout_handler (need clean up later)

* cleaup

* more in inproc_timeout_handler

* fix for linux build

* more fixes for unix, fmt

* revert timeoutexecutor api

* revert baby_fuzzer/src/main.rs

* various fixes

* no unsafe

* remove timer in crash_handler
2021-08-18 09:11:34 +02:00
Andrea Fioraldi
5cd7339b1a
Qemu read memory mappings (#228)
* add x64 syscalls numbers

* syscall hook

* update commit

* read guest mappings

* clippy
2021-07-16 10:38:00 +02:00
Andrea Fioraldi
b09fa4e3f4
Qemu syscalls hook (#226)
* add x64 syscalls numbers

* syscall hook

* update commit
2021-07-15 11:12:10 +02:00
Andrea Fioraldi
ec4c5ae88a
remove timeout on crash (#224) 2021-07-13 16:15:12 +02:00
Dominik Maier
0121096e84
Fixes for no_std build (#214)
* builds on no_std

* fixed std build

* nightly fmt on CI

* nightly fmt on CI (again)

* fmt

* no_std build on unix

* more mem

* added no_std from #212 to gh workflow

* more ci, less nightly

* clippy

* more toolchains?

* docu

* y u no build

* more ci?

* next try

* fixed dockr

* more dockerfile fixes

* ondisk corpus fixed

* panic:?

* ubunutu

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-09 20:07:56 +02:00
Andrea Fioraldi
bdb5efbf5b
Configurations (#162)
* print sender id

* storing sender id to env

* executor in llmp handle_in_client

* compile the lib

* compiling generic_inmemory

* fix forkserver

* adapt from fuzzers

* instrospection fix

* exitkind in NewTestcase

* fix libafl_frida

* fix firda_libpng

* send conf with Newtestcase event

* bump to 0.4.0

* no_std fix

* fmt

* fix libfuzzer_libmozjpeg

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-06-22 15:04:14 +02:00
Toka
e68eaf8244
Timeout forkserver (#136)
* barebones for TimeoutForkserverExecutor

* TimeoutForkserverExecutor

* update pid in forkserverexecutor

* clppy and other small fixes

* doc

* fix

* no unwrap

* read_exact and error handling

* fix

* semicolon
2021-05-30 23:11:10 +02:00
Toka
5d44846c27
Improve TimeoutExecutor (#126)
* improve TimeoutExecutor

* fix for windows build
2021-05-25 22:20:52 +02:00
Andrea Fioraldi
46716e8090
Remove executor hooks (#124)
* remove HasExecHooks from Executor

* adapt the frida executor

* adapt frida and avoid recursive type infearence

* fix win build
2021-05-25 15:19:10 +02:00
Andrea Fioraldi
b51936397b
Introduce FeedbackState and allow Feedbacks to process the entire State (#103)
* save work

* it builds

* MutationalStage builds

* compile lib.rs test

* libafl tests work

* adapt stb_image example

* change fuzzer to not hold executor and event manager as type field

* libfuzzer_stb_image running example

* restore ReachabilityFeedback

* restore introspection

* adapt fuzzers except frida_libpng

* format

* compile on windows

* clippy

* fix libafl_frida

* adapt frida_libpng
2021-05-12 23:53:27 +02:00
Dominik Maier
08a32c3856 more docs 2021-05-06 20:10:45 +02:00
Andrea Fioraldi
9e9d95f93d Observers refactor (#84)
* new observer structure with HasExecHooks

* adapt libafl_frida to new observers

* docstrings
2021-05-06 20:10:45 +02:00
s1341
5c856cccc8
WIP: Implement ASAN instrumentation using frida (#45)
* frida_asan: Implemented initial asan runtime library

* frida_asan: Switch to hashbrown

* Implemented GOT-based hooking to isolate the hooking of the memory functions. Implemented initial ASAN instrumentation

* WIP: Shadowing all used memory. Currently tracking pages using a BTreeSet. Slow AF!

* Add SigTrap to unix_signals and inprocess

* Working frida-asan, almost no speed degradation.

Currently the shadow check is reversed, so it checks only that the shadow is not 0.
We need to implement sub-8-byte checking.

* Format

* Cleanup and formatting

* Sub-qword and 16-byte checks implemented; Fixed unaligned access to QWORD

* Pass the ucontext_t to signal handlers. Initial regdump on crash

* Fix typo

* Make the context argument a mut ref

* Add missing files; Implement initial reporting

* Refactor out gothook; Move safety checkers to dynasm

* Get rid of const assembly blobs no longer needed

* Move to a handler function instead of using SIGTRAP.

This bloats the transformed code, but doesn't seem to have a major impact on performance.

Also, implemented pretty backtraces and assembly output.

* Formatting

* Get rid of all the pinning crap I wasted my day on, We don't need it

* windows fixes

* ashmem

* ashmem_service: server side ready

* ashmem_service: client side ready. Ready for integration

* ashmem_service: changes to UnixShMem to make it 'threadable'

* ashmem_service: format

* ashmem_service: Undo changes to UnixShMem, make the thread own the AshmemService instead; Fix protocol bug

* ashmem_service: working ashmem service. Fix merge issues

* use the newly released capston e 0.8.0; Fix a nasty bug where the afl_area an pc_pointer were reversed. Changed Vectors to Boxed [u8]

* Implement type detection for reporting; Implement double-free/unallocated free checking

* fmt

* Cleanup code a little

* frida-asan: This is an omnibus commit. Should probably have been a bunch of small commits, but I don't have the time/patience.

 - Implemented DrCov support in order to debug a failing harness. This is actually
   generic and should be moved out of libafl_frida.
 - Implemented LIBAFL_FRIDA_OPTIONS env var to pass options to the frida helper,
   to dynamically enable/disable asan and drcov.
 - Implemented memory reuse - after each test case the used pages are recycled and
   can be reused in the next test case.
 - Implemented and tested vectorized instruction instrumentation.
 - Implemented not instrumenting atomic load/store instructions. The cost of
   trying to emulate their behaviour is too high at the moment.
 - Implemented probing of shadow bit to determine the best match for the current
   system.
 - Implemented shadow memory pre-mapping where it is available. We probe for this
   too.
 - Implemented ability to specify a list of modules to instrument on the command
   line. This allows fine-grained control of which modules are instrumented for
   coverage/asan/drcov.
 - Implemented unpoisoning of the Input target_bytes in a pre_exec hook.
 - Added support for zero-sized allocations. We return 0x10 bytes at the moment.
 - Added all known operator new/delete functions to hooks.
 - Added workaround for frida_gum_allocate_near bug.
 - Cleaned up reporting, added reporting for different error types.

* frida-asan: Implement leak detection

* Fix merge issues

* Rebased on dev to get llmp/shmem changes; Clippy fixes

* Add FridaOptions struct

* Add the Custom ExitKind; Get rid of Clone/PartialEq on ExitKind

* Make it possible to recover from an ASAN error

* Add SIGTRAP to crashing signals

* Add back (conditional) crashing on Asan errors.

* Fix too-large immediates in add instruction

* Implement RcShMemProvider, finally fix the EOP bug

* Clear ASAN_ERRORS before each test

* Fix warnings; Fix review issues

* Cleanup prints

* Add timeout to Frida mode

* Make allocation-/free-site backtraces optional

* CPU Context and backtrace (on android/aarch64 atm) on crash

* Make stalker conditional

* Add metadata to solution, and write metadata files

* Add addresses to backtrace; Add reporting of ASAN stack errors; Fix ASAN reporting bugs

* Remove meaningless backtrace on crash

* Fix the x0, x1 load in report

* use upstream color-backtrace

* use __builtin_thread_pointer instead of custom asm

* Don't unwrap ASAN_ERRORS if it isn't some

* Fix bug where we weren't clearing the drcov basicblocks after each run

* Fix bug where we were dropping an ashmem too soon

* Fix OwnedPtr instead of CPtr

* Fix gettls for all archs

* cfg guards for target arch, disabling Frida-ASAN/-DrCov if not on aarch64

* Cargo fmt

* Only panic in options when asan/drcov are turned on; Merge fixes

* gothook only supported on unix

* Fix gettls on msvc

* Another attempt to fix MSVC gettls

* Fix backtrace use

* nostd fixes; warning fixes

* formatting

* Migrate FridaEdgeCoverageHelper into libafl_frida, and rename to FridaInstrumentationHelper

* Clean up uses

* Move DrCovWriter to libafl_targets

* Refactor DrCovWriter to get a vec of DrCovBasicBlocks; formatting

* Update to newer backtrace which supports android with gimli

* windows fixes

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: andreafioraldi <andreafioraldi@gmail.com>
2021-04-28 10:12:49 +02:00
Dominik Maier
d72d48d6a8 more cargo fixes 2021-04-07 15:57:59 +02:00
root
a0550b3154 compile libfuzzer_libpng on windows 2021-03-25 18:50:40 +01:00
Dominik Maier
556141e9a3 ignored frida on windows 2021-03-22 16:54:31 +01:00
Andrea Fioraldi
3f013c481f windows_exception_handler in InProcessExecutor 2021-03-18 17:22:47 +01:00
Andrea Fioraldi
8a3e4301fb format 2021-03-17 17:19:33 +01:00
Andrea Fioraldi
2b2599833c fix timeout executor no_std 2021-03-17 17:14:46 +01:00
Andrea Fioraldi
cd7030443e timeout executor file 2021-03-17 17:08:17 +01:00