191 Commits

Author SHA1 Message Date
Toka
7f4e341741
inprocessfork executor (#237)
* inprocessfork executor

* fmt

* cfg

* no_std

* no volatile rw

* wrapping_add

* fix

* mutable pointer

* ptr initialization in __sanitizer_cov_trace_pc_guard_init

* features

* more cfg

* fmt

* fix

* fmt

* post_fork

* fmt

* pre_fork

* test

* cfg
2021-08-07 12:09:54 +02:00
Dominik Maier
16c3a07be7
ShMem Server for MacOS (#238)
* generalized ashmem server

* fixed macos testcases

* added StdShMemService

* no_st

* fmt

* added testcase, fixed some bugs (not all)

* solidified unix shmem

* initial impl for MmapShMem

* Added shmem service start to more testcases

* clippy

* fixed tetcases

* added frida_libpng makefile for easy use

* trying to fix build on ubuntu

* fixed ubuntu build for libpng

* no_std

* fixed testcase
2021-08-05 17:08:01 +02:00
julihoh
704830a501
Documentation and Refactors for Concolic (#243)
* document symcc_runtime

* rename serialization format expressions to be more concise

* authorship notes

* document dump_constraints

* document smoke test

* tests for serialization format and refactoring

* remove unused bswap message

* remove obselete SymExpr::End

* document and refactor serialization format

* fmt

* more missing docs

* typos

* clippy
2021-08-05 15:01:14 +02:00
julihoh
3d98d31712
Concolic Tracing (#160)
* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* fixed build, clippy

* no_std

* implement WithObservers executor as discussed

* add symqemu as a submodule

* fix symqemu submodule URL to be relative

* update the concolic runtime to match the new interface

* update the trace file header regularly to save constraints in case the program crashes

* add build dependencies for symqemu

* handle full mesage buffer properly

* better policy for updating trace header

* less aggregiously inefficient GC information serialization

* move concolic runtime hitmap count to filter

this is in preparation for the new runtime interface

* very WIP new runtime interface

* use more convenient types in rust runtime

* EmptyRuntime -> NopRuntime

* hide cpp_runtime and formatting

* implement tracing runtime using new runtime interface

* implement filters with new runtime interface

* use a local checkout for symcc_runtime

* make test runtime tracing

* use test_runtime in smoke test

* fix formatting

* make the clippy overlord happy?

* disable symcc build on everything but linux

* make more of symcc_runtime linux only

* fix linking symcc_runtime with C++ stdlib

* will clippy ever be happy?

* formatting

* don't export symcc runtime when compiling tests

* clippy...

* "don't export symcc runtime when compiling tests" for runtime crate as well

* clippy

* move command executor to LibAFL

* move concolic crate into LibAFL

* move concolic{metada,observer} into LibAFL

* move ConcolicFeedback into LibAFL

* move ConolicStage into LibAFL

* fix bug in symcc part of concolic runtime

* stb_image fuzzer with concolic as example fuzzer

* clean up basic_concolic_fuzzer

* clean up and document concolic example fuzzer

* formatting

* clippy

* remove basic_concolic_fuzzer (it is now part of the examples)

* remove the runtime crate in favor of symcc_runtime

* re-architect concolic smoke test and remove git submodules

* remove old submodule directories

* make coverage filter public

* focker docker build

* clippy

* clippy fixes

* fix ubuntu as well

* remove .gitmodules

* move concolic mutational stage into libafl behind feature flag

* script to install dependencies for concolic smoke test

* fix bug

* clippy

* add github action to run smoke test

* fix action

* ensure smoke test is run in correct directory

* remove devcontainer files

* address feedback

* clippy

* more clippy

* address more feedback

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-08-05 13:22:00 +02:00
David CARLIER
b0cb74324c
fix android arm64 build (#231) 2021-07-20 02:38:44 +02:00
Andrea Fioraldi
5cd7339b1a
Qemu read memory mappings (#228)
* add x64 syscalls numbers

* syscall hook

* update commit

* read guest mappings

* clippy
2021-07-16 10:38:00 +02:00
Andrea Fioraldi
b09fa4e3f4
Qemu syscalls hook (#226)
* add x64 syscalls numbers

* syscall hook

* update commit
2021-07-15 11:12:10 +02:00
Andrea Fioraldi
ec4c5ae88a
remove timeout on crash (#224) 2021-07-13 16:15:12 +02:00
Dominik Maier
0121096e84
Fixes for no_std build (#214)
* builds on no_std

* fixed std build

* nightly fmt on CI

* nightly fmt on CI (again)

* fmt

* no_std build on unix

* more mem

* added no_std from #212 to gh workflow

* more ci, less nightly

* clippy

* more toolchains?

* docu

* y u no build

* more ci?

* next try

* fixed dockr

* more dockerfile fixes

* ondisk corpus fixed

* panic:?

* ubunutu

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-09 20:07:56 +02:00
Andrea Fioraldi
d472a1242a
libafl_qemu (#211)
* empty libafl_qemu crate

* fuzzbench qemu fuzzer skeleton

* emu.run() works without bp

* working emu loop

* resolve elf symbols

* running Qemu fuzzer without coverage

* qemu fuzzer with edge coverage

* merge into inprocess::GLOBAL_STATE

* create QemuExecutor and remove QemuEmulator

* qemu hooks and persist edges mapping storing them in State

* windows fix

* add libafl_qemu to workspace

* windows fix

* some clippy

* clippy

* fix fuzzbench_qemu

* fix fuzzbench_qemu makefile

* fuck you macos
2021-07-08 15:21:14 +02:00
Andrea Fioraldi
44f6e4c389
Improve introspection (#200)
* remove NUM_FEEDBACKS

* working introspection

* adust introspection stats

* bugfixes, clippy

* removed outdated define

* more clippy;

* no_std

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-07-02 10:58:36 +02:00
Andrea Fioraldi
bdb5efbf5b
Configurations (#162)
* print sender id

* storing sender id to env

* executor in llmp handle_in_client

* compile the lib

* compiling generic_inmemory

* fix forkserver

* adapt from fuzzers

* instrospection fix

* exitkind in NewTestcase

* fix libafl_frida

* fix firda_libpng

* send conf with Newtestcase event

* bump to 0.4.0

* no_std fix

* fmt

* fix libfuzzer_libmozjpeg

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-06-22 15:04:14 +02:00
Dominik Maier
5c5a1cf2e9
fixed fuzzbench early exits (#182) 2021-06-22 09:02:56 +02:00
Andrea Fioraldi
fee9cae8ed Fix InProcessExecutor test mod 2021-06-21 11:58:04 +02:00
Andrea Fioraldi
6351cb16e5 Support multiple crash handlers when using multiple InProcessExecutors 2021-06-21 11:39:42 +02:00
Dominik Maier
dea21da5c3
fuzzbench harness (#165)
* starting to build fuzzbench harness

* fuzzbench updated

* fuzzbench example readme

* removed dummy files

* Intial de-luxe dockerfile added

* added to dockerignore

* more fuzzbench

* dockerfile

* final dockerfile fun

* fuzzing fixes, switched rand, build fixes

* fmt

* added dummy fuzzone

* silence wrapper output

* clippy

* logfile fixes

* adopt changes to libafl-cc

* various fixes
2021-06-16 18:24:07 +02:00
Gal Tashma
8202548648
dont send was_killed to forkserver more than once (#175)
Co-authored-by: galtashma <gal.tashma@swg.local>
2021-06-15 21:58:05 +02:00
Dominik Maier
c2feddbe7c
Implemented rand::Rng for libafl::Rand as feature flag (#174)
* implemented rand::Rng for libafl::Rand as feature flag

* fixed build

* renamed macro
2021-06-15 09:30:32 +02:00
Dominik Maier
c123872b11 more informative error message for forkserver 2021-06-10 22:19:44 +02:00
Andrea Fioraldi
8fa654dd61 fix generics for ShadowExecutor 2021-06-09 14:34:38 +02:00
Andrea Fioraldi
2b16e92461 introduce ShadowTracingStage 2021-06-09 14:17:43 +02:00
Andrea Fioraldi
9e9425c622 introduce ShadowExecutor 2021-06-09 12:15:08 +02:00
Dominik Maier
36b823548a
nightly clippy fixes (#155)
* nightly clippy fixes

* more niglty clippy fixes

* added Safety section

* no_std fixes

* final fixes
2021-06-07 12:30:56 +02:00
Dominik Maier
3b2ee4bb70
Added MacOS CI (#131)
* added macos ci

* running tests on macos

* some macos fixes

* fmt

* some must_use infos

* trying' to fix MacOs testcases

* no main in test

* fixed MacOS testcases

* tried to fix build errors

* unified shmem_limits

* Revert "unified shmem_limits"

This reverts commit 8c6bb8d6a2cec71d72bb181b5b491737a771298e.

* hopefully fixed macos testcase

* removed unneeded values
2021-06-07 01:24:41 +02:00
s1341
636194de0e
Frida switch from walk-proc-maps to frida-gum based extraction of ranges (#149)
* Bump frida-gum version

* Move from walk of /proc/pid/maps to frida based range/module locatoin
2021-06-06 10:40:07 +02:00
Toka
c90604f123
TimeoutForkserverExecutor fix (#140)
* fix the call to kill

* fix
2021-05-31 19:31:32 +02:00
Toka
e4b3cc542a
Forkserver_simple uses TimeoutForkserverExecutor (#139)
* forkserver_simple uses TimeoutforkserverExecutor

* fmt

* from_millis
2021-05-31 15:44:56 +02:00
Toka
e68eaf8244
Timeout forkserver (#136)
* barebones for TimeoutForkserverExecutor

* TimeoutForkserverExecutor

* update pid in forkserverexecutor

* clppy and other small fixes

* doc

* fix

* no unwrap

* read_exact and error handling

* fix

* semicolon
2021-05-30 23:11:10 +02:00
David CARLIER
e61f80584e
FreeBSD build fix proposal. (#137)
rlim_t is signed in this platform.
2021-05-29 00:31:04 +02:00
Dominik Maier
81ccf7f7d8 some macos fixes 2021-05-26 13:58:12 +02:00
Toka
5d44846c27
Improve TimeoutExecutor (#126)
* improve TimeoutExecutor

* fix for windows build
2021-05-25 22:20:52 +02:00
Andrea Fioraldi
46716e8090
Remove executor hooks (#124)
* remove HasExecHooks from Executor

* adapt the frida executor

* adapt frida and avoid recursive type infearence

* fix win build
2021-05-25 15:19:10 +02:00
s1341
3a21ad59a4
Hook using frida gum interceptor instead of gothook (#112)
* Move from gothook to frida-based hooks

* Force link against libc++

* Clippy + cleanup prints

* exclude ranges

* Add back guard pages; Implement libc hooks

* Bump frida-rust version

* Add hooks for mmap/munmap, as per issue #105

* Refactor to get rid of global allocator singleton

* Cleanup imports; Fix free out-of-range; Move to fixed addresses for asan allocatoins

* use frida-rust from crates.io now that it has caught up

* cargo fmt

* Clippy fixes

* Better clippy fix

* More clippy fix

* Formatting

* Review changes
2021-05-25 13:45:06 +02:00
Toka
d4410c072a
ForkserverExecutor (#111)
* add Forkserver, Pipe Outfile struct

* add forkserver executor struct, and shmem init

* close pipes in the destructor of Forkserver

* fill pre_exec to write out the inputs

* fix

* read_st, write_ctl

* more handshakes

* wrap Pipe in Arc, fill post_exec

* add Forkserver, Pipe Outfile struct

* add forkserver executor struct, and shmem init

* close pipes in the destructor of Forkserver

* fill pre_exec to write out the inputs

* fix

* read_st, write_ctl

* more handshakes

* wrap Pipe in Arc, fill post_exec

* fix for the lastest HasExecHooks trait

* use Dominik's pipe, remove Arc and temporarily pass RawFd to setstdin but trying to figure out other solutions

* add libafl_tests, put a very simple vulnerable program

* fix

* added forkserver_simple (mostly copy-pasted from babyfuzzer)

* fix test

* handle crash in post_exec

* add README.md

* check exec time to see why it's so slow

* remove double invokation of is_interesting for the obejctive

* make forkserver_simple AFL-like and improve speed

* some debugging help

* do not evaluate feedback if solution

* speedup the things

* working input placement via stdin in Forkserver

* don't call panic! but return errors, rewrite some comments

* use AFLplusplus/afl-cc instead of AFL

* use .cur_input like AFL

* bring the test for forkserver back

* add better README.md message

* failing the initial handshake should return an error

* delete some commented-out code

* format

* format

* ForkserverExecutor needs std and is unix-only for now

* clippy

* OutFile error handling

* fmt

* clippy

* don't build libafl_tests on windows

* fix

* keep test in forkserver.rs simple

* add forkserver_test feature for libafl_tests

* format

* some doc

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-05-25 13:40:00 +02:00
Max Ammann
bfbaa7ae83
Make InProcessExecutor use a plain Input instead of bytes (#115)
* Make InProcessExecutor use a plain Input instead of bytes

* Use HasTargetBytes and fix CI
2021-05-20 17:57:31 +02:00
Andrea Fioraldi
2f54e9dc01
UserStats (#114)
* MultiStats

* custom event in MapFeedback

* fix introspection

* fix windows

* clippy

* fix nostd

* bump to 0.3.2
2021-05-20 16:49:12 +02:00
Andrea Fioraldi
acc5ed42a2
Basic CmpLog (#113)
* libafl_targets: refactor sancov trace-pc

* cmp observer

* libaf_targets: new structure to isolate sancov

* fix C warning

* combined executor

* cmp observer and feedback

* I2SRandReplace mutator

* impl CmpMap for CmpLogMap in libafl_targets

* cmplog observer

* clippy

* TracingStage

* working random cmplog mutations

* enable cmplog for libfuzzer_stb_image

* re-enable new testcase stats print

* fix update stats display

* bump 0.3.1

* clippy

* clippy

* no clippy for fuzzers/

* fix

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-05-20 14:05:20 +02:00
Vimal Joseph
d991395c81
Launcher (#48)
* launcher in linux

* silence stdout and stderr linux

* arg parser and other changes

* retry instead of sleep

* no_std fixes

* reordered includes

* launcher for windows and kill clients when broker returns

* cargo fmt

* started launcher api cleanup

* use closures instead of functions

* small change

* reordered launcher params

* fixed clippy warnings

* fixed no_std

* moved launcher example to own folder

* docu

* cleanup launcher

* more docs

* Fix merge issues

* Rework the launcher code to provide a cleaner API

* Open file before spawning clients

* launcher: fix merge issue, sleep for a different amount for each core

* fixed no_std

* Tcp Broker to Broker Communication (#66)

* initial b2b implementation

* no_std and clippy fixes

* b2b testcase added

* more correct testcases

* fixed b2b

* typo

* fixed unused warning

* some clippy warning ignored

* using clippy.sh

* Update README.md

* fixed clippy run in workflow

* fixing clippy::match-same-arms

* make clippy less pedantic

* fixed some minor typos in the book

* launcher: use s1341's fork of core_affinity

* Build warning fix proposal, mostly about reference to packed fields. (#79)

* Observers refactor (#84)

* new observer structure with HasExecHooks

* adapt libafl_frida to new observers

* docstrings

* Composing feedback (#85)

* composing feedbacks as logic operations and bump to 0.2

* adapt fuzzers and libafl_frida

* fix windows build

* fixed clippy warnings

* Frida suppress instrumentation locations option (#87)

* Implement  frida option

* Format

* add append/discard_metadata for and/or/not feedback (#86)

* add append/discard_metadata for and/or/not feedback

* fix

* Call append_metadata on crash (#88)

* Call append_metadata on crash

* Formatting

* Reachability example (#65)

* add reachability observer/feedback

* add fuzzer exmaple

* fmt

* remove reachabilityobserver, use stdmapobserver instead

* update diff.patch

* update README

* fix the clippy warning

* Squashed commit of the following:

commit f20524ebd77011481e86b420c925e8504bd11308
Author: Andrea Fioraldi <andreafioraldi@gmail.com>
Date:   Tue May 4 16:00:39 2021 +0200

    Composing feedback (#85)

    * composing feedbacks as logic operations and bump to 0.2

    * adapt fuzzers and libafl_frida

    * fix windows build

commit e06efaa03bc96ef71740d7376c7381572bf11c6c
Author: Andrea Fioraldi <andreafioraldi@gmail.com>
Date:   Tue May 4 13:54:46 2021 +0200

    Observers refactor (#84)

    * new observer structure with HasExecHooks

    * adapt libafl_frida to new observers

    * docstrings

commit 17c6fcd31cb746c099654be2b7a168bd04d46381
Merge: 08a2d43 a78a4b7
Author: Andrea Fioraldi <andreafioraldi@gmail.com>
Date:   Mon May 3 11:16:49 2021 +0200

    Merge branch 'main' into dev

commit 08a2d43790797d8864565fec99e7043289a46283
Author: David CARLIER <devnexen@gmail.com>
Date:   Mon May 3 10:15:28 2021 +0100

    Build warning fix proposal, mostly about reference to packed fields. (#79)

commit 88fe8fa532ac34cbc10782f5f71264f620385dda
Merge: d5d46ad d2e7719
Author: Andrea Fioraldi <andreafioraldi@gmail.com>
Date:   Mon May 3 11:05:42 2021 +0200

    Merge pull request #80 from marcograss/book-typos

    fixed some minor typos in the book

commit a78a4b73fa798c1ed7a3d053369cca435e57aa07
Author: s1341 <s1341@users.noreply.github.com>
Date:   Mon May 3 10:34:15 2021 +0300

    frida-asan: Un-inline report funclet to reduce code bloat (#81)

    * frida-asan: Outline report funclet to reduce code bloat

    * fmt

commit d2e7719a8bea3a993394c187e2183d3e91f02c75
Author: Marco Grassi <marco.gra@gmail.com>
Date:   Sun May 2 21:58:33 2021 +0800

    fixed some minor typos in the book

commit d5d46ad7e440fd4a2925352ed1ccb9ced5d9463d
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 23:09:10 2021 +0200

    make clippy less pedantic

commit 52d25e979e23589587c885803641058dc36aa998
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 22:23:59 2021 +0200

    fixing clippy::match-same-arms

commit cd66f880dea830d1e38e89fd1bf3c20fd89c9d70
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 14:02:07 2021 +0200

    fixed clippy run in workflow

commit ddcf086acde2b703c36e4ec3976588313fc3d591
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 13:53:29 2021 +0200

    Update README.md

commit c715f1fe6e42942e53bd13ea6a23214620f6c829
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 13:48:38 2021 +0200

    using clippy.sh

commit 9374b26b1d2d44c6042fdd653a8d960ce698592c
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 13:47:44 2021 +0200

    some clippy warning ignored

commit b9e75c0c98fdfb1e70778e6f3612a94b71dcd21a
Author: Dominik Maier <domenukk@gmail.com>
Date:   Sat May 1 13:24:02 2021 +0200

    Tcp Broker to Broker Communication (#66)

    * initial b2b implementation

    * no_std and clippy fixes

    * b2b testcase added

    * more correct testcases

    * fixed b2b

    * typo

    * fixed unused warning

* feedbacks now return a boolean value

* use feedback_or, and modify Cargo.toml

* fix diff between dev and this branch

* fmt

Co-authored-by: Dominik Maier <domenukk@gmail.com>

* clippy fixes

* clippy fixes

* clippy fixes, x86_64 warnings

* more docs

* Observers lifetime (#89)

* introduce MatchName and alow lifetimes in observers

* adapt fuzzers to observers with lifetime

* introduce type_eq when on nightly

* fix no_std

* fmt

* Better docu (#90)

* more docs

* more docs:

* more docu

* more docu

* finished docs

* cleaned up markup

* must_use tags added

* more docs

* more docu, less clippy

* more fixes

* Clippy fixes (#92)

* more docs

* more docs:

* more docu

* more docu

* finished docs

* cleaned up markup

* must_use tags added

* more docs

* swapped if/else, as per clippy

* more docu, less clippy

* more fixes

* Fix merge issues

* Get rid of unneeded prints

* Fix merge errors

* added b2b to restarting interface

* Setting SO_REUSEPORT

* added b2b to launcher api

* more windows launcher

* Fix merge errors

* Add b2b support to frida_libpng

* make frida_libpng bind to a public address

* Convert launcher into a builder LauncherBuilder

* formatting

* Convert setup_restarting_mgr to a builder RestartingMgrBuilder; leave setup_restarting_mgr_std as is, so that fuzzers work

* RcShmem should be locked via a mutex

* Wait at least 1 second between broker and first client, to avoid race

* update frida_libpng README for cross-compiling to android (#100)

Co-authored-by: Ariel Zentner <ArielZ@nsogroup.com>

* Fixed build for Windows

* no_std fixes

* reverted aa6773dcade93b3a66ce86e6b2cc75f55ce194e7 & windows fixes

* added pipes, moving to remove race conditions for rc shmem

* fix unix build

* fixed clippy:

* fixed no_std once more

* renamed b2b to remote_broker_addr

* you get a pre_fork, and you get a post_fork, forks for everyone

* switched to typed_builder

* Fix merge isseu

* Fix frida fuzzer with new Launcher builder

* Introspection (#97)

* Rework to put `ClientPerfStats` in `State` and pass that along. Still need to work on getting granular information from `Feedback` and `Observer`

* Add perf_stats feature to libafl/Cargo.toml

* Update feedbacks to have with_perf

* Remove unneeeded print statement

* cargo fmt all the things

* use local llvmint vs cpu specific asm for reading cycle counter

* Remove debug testing code

* Stats timeout to 3 seconds

* Inline smallish functions for ClientPerfStats

* Remove .libs/llvmint and have the correct conditional compilation of link_llvm_intrinsics on the perf_stats feature

* pub(crate) the NUM_FEEDBACK and NUM_STAGES consts

* Tcp Broker to Broker Communication (#66)

* initial b2b implementation

* no_std and clippy fixes

* b2b testcase added

* more correct testcases

* fixed b2b

* typo

* fixed unused warning

* clippy fixes

* fallback to systemtime on non-x86

* make clippy more strict

* small fixes

* bump 0.2.1

* readme

Co-authored-by: ctfhacker <cld251@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>

* typos (please review)

* merged clippy.sh

* utils

* Add asan cores option (#102)

* added asan-cores option for frida fuzzer

When asan is enabled (via LIBBAFL_FRIDA_OPTIONS enable-asan), you can
filter exactly which of the cores asan should run on with the
asan-cores variable.

* add is_some check instead of !None

Co-authored-by: Ariel Zentner <ArielZ@nsogroup.com>

* moved utils to bolts

* fixed typo

* no_std fixes

* unix fixes

* fixed unix no_std build

* fix llmp.rs

* adapt libfuzzer_libpng_launcher

* added all fuzzers to ci

* fmt, improved ci

* tests crate not ready for prime time

* clippy fixes

* make ci script executable

* trying to fix example fuzzers

* working libfuzzer_libpng_laucnher

* frida_libpng builds

* clippy

* bump version

* fix no_std

* fix dep version

* clippy fixes

* more fies

* clippy++

* warn again

* clearer readme

Co-authored-by: Vimal Joseph <vimaljoseph027@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: s1341 <github@shmarya.net>
Co-authored-by: Marco Grassi <marco.gra@gmail.com>
Co-authored-by: s1341 <s1341@users.noreply.github.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: Toka <tokazerkje@outlook.com>
Co-authored-by: r-e-l-z <azentner@gmail.com>
Co-authored-by: Ariel Zentner <ArielZ@nsogroup.com>
Co-authored-by: ctfhacker <cld251@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
2021-05-19 13:08:24 +02:00
Andrea Fioraldi
b51936397b
Introduce FeedbackState and allow Feedbacks to process the entire State (#103)
* save work

* it builds

* MutationalStage builds

* compile lib.rs test

* libafl tests work

* adapt stb_image example

* change fuzzer to not hold executor and event manager as type field

* libfuzzer_stb_image running example

* restore ReachabilityFeedback

* restore introspection

* adapt fuzzers except frida_libpng

* format

* compile on windows

* clippy

* fix libafl_frida

* adapt frida_libpng
2021-05-12 23:53:27 +02:00
Dominik Maier
7cd99543bb Clippy fixes (#92)
* more docs

* more docs:

* more docu

* more docu

* finished docs

* cleaned up markup

* must_use tags added

* more docs

* swapped if/else, as per clippy

* more docu, less clippy

* more fixes
2021-05-06 20:10:45 +02:00
Dominik Maier
ed9169119e Better docu (#90)
* more docs

* more docs:

* more docu

* more docu

* finished docs

* cleaned up markup

* must_use tags added

* more docs

* more docu, less clippy

* more fixes
2021-05-06 20:10:45 +02:00
Dominik Maier
08a32c3856 more docs 2021-05-06 20:10:45 +02:00
s1341
2e192fd6cc Call append_metadata on crash (#88)
* Call append_metadata on crash

* Formatting
2021-05-06 20:10:45 +02:00
Andrea Fioraldi
9f3b0984c3 Composing feedback (#85)
* composing feedbacks as logic operations and bump to 0.2

* adapt fuzzers and libafl_frida

* fix windows build
2021-05-06 20:10:45 +02:00
Andrea Fioraldi
9e9d95f93d Observers refactor (#84)
* new observer structure with HasExecHooks

* adapt libafl_frida to new observers

* docstrings
2021-05-06 20:10:45 +02:00
Dominik Maier
8717d07369 some clippy warning ignored 2021-05-06 20:10:45 +02:00
s1341
5c856cccc8
WIP: Implement ASAN instrumentation using frida (#45)
* frida_asan: Implemented initial asan runtime library

* frida_asan: Switch to hashbrown

* Implemented GOT-based hooking to isolate the hooking of the memory functions. Implemented initial ASAN instrumentation

* WIP: Shadowing all used memory. Currently tracking pages using a BTreeSet. Slow AF!

* Add SigTrap to unix_signals and inprocess

* Working frida-asan, almost no speed degradation.

Currently the shadow check is reversed, so it checks only that the shadow is not 0.
We need to implement sub-8-byte checking.

* Format

* Cleanup and formatting

* Sub-qword and 16-byte checks implemented; Fixed unaligned access to QWORD

* Pass the ucontext_t to signal handlers. Initial regdump on crash

* Fix typo

* Make the context argument a mut ref

* Add missing files; Implement initial reporting

* Refactor out gothook; Move safety checkers to dynasm

* Get rid of const assembly blobs no longer needed

* Move to a handler function instead of using SIGTRAP.

This bloats the transformed code, but doesn't seem to have a major impact on performance.

Also, implemented pretty backtraces and assembly output.

* Formatting

* Get rid of all the pinning crap I wasted my day on, We don't need it

* windows fixes

* ashmem

* ashmem_service: server side ready

* ashmem_service: client side ready. Ready for integration

* ashmem_service: changes to UnixShMem to make it 'threadable'

* ashmem_service: format

* ashmem_service: Undo changes to UnixShMem, make the thread own the AshmemService instead; Fix protocol bug

* ashmem_service: working ashmem service. Fix merge issues

* use the newly released capston e 0.8.0; Fix a nasty bug where the afl_area an pc_pointer were reversed. Changed Vectors to Boxed [u8]

* Implement type detection for reporting; Implement double-free/unallocated free checking

* fmt

* Cleanup code a little

* frida-asan: This is an omnibus commit. Should probably have been a bunch of small commits, but I don't have the time/patience.

 - Implemented DrCov support in order to debug a failing harness. This is actually
   generic and should be moved out of libafl_frida.
 - Implemented LIBAFL_FRIDA_OPTIONS env var to pass options to the frida helper,
   to dynamically enable/disable asan and drcov.
 - Implemented memory reuse - after each test case the used pages are recycled and
   can be reused in the next test case.
 - Implemented and tested vectorized instruction instrumentation.
 - Implemented not instrumenting atomic load/store instructions. The cost of
   trying to emulate their behaviour is too high at the moment.
 - Implemented probing of shadow bit to determine the best match for the current
   system.
 - Implemented shadow memory pre-mapping where it is available. We probe for this
   too.
 - Implemented ability to specify a list of modules to instrument on the command
   line. This allows fine-grained control of which modules are instrumented for
   coverage/asan/drcov.
 - Implemented unpoisoning of the Input target_bytes in a pre_exec hook.
 - Added support for zero-sized allocations. We return 0x10 bytes at the moment.
 - Added all known operator new/delete functions to hooks.
 - Added workaround for frida_gum_allocate_near bug.
 - Cleaned up reporting, added reporting for different error types.

* frida-asan: Implement leak detection

* Fix merge issues

* Rebased on dev to get llmp/shmem changes; Clippy fixes

* Add FridaOptions struct

* Add the Custom ExitKind; Get rid of Clone/PartialEq on ExitKind

* Make it possible to recover from an ASAN error

* Add SIGTRAP to crashing signals

* Add back (conditional) crashing on Asan errors.

* Fix too-large immediates in add instruction

* Implement RcShMemProvider, finally fix the EOP bug

* Clear ASAN_ERRORS before each test

* Fix warnings; Fix review issues

* Cleanup prints

* Add timeout to Frida mode

* Make allocation-/free-site backtraces optional

* CPU Context and backtrace (on android/aarch64 atm) on crash

* Make stalker conditional

* Add metadata to solution, and write metadata files

* Add addresses to backtrace; Add reporting of ASAN stack errors; Fix ASAN reporting bugs

* Remove meaningless backtrace on crash

* Fix the x0, x1 load in report

* use upstream color-backtrace

* use __builtin_thread_pointer instead of custom asm

* Don't unwrap ASAN_ERRORS if it isn't some

* Fix bug where we weren't clearing the drcov basicblocks after each run

* Fix bug where we were dropping an ashmem too soon

* Fix OwnedPtr instead of CPtr

* Fix gettls for all archs

* cfg guards for target arch, disabling Frida-ASAN/-DrCov if not on aarch64

* Cargo fmt

* Only panic in options when asan/drcov are turned on; Merge fixes

* gothook only supported on unix

* Fix gettls on msvc

* Another attempt to fix MSVC gettls

* Fix backtrace use

* nostd fixes; warning fixes

* formatting

* Migrate FridaEdgeCoverageHelper into libafl_frida, and rename to FridaInstrumentationHelper

* Clean up uses

* Move DrCovWriter to libafl_targets

* Refactor DrCovWriter to get a vec of DrCovBasicBlocks; formatting

* Update to newer backtrace which supports android with gimli

* windows fixes

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: andreafioraldi <andreafioraldi@gmail.com>
2021-04-28 10:12:49 +02:00
Dominik Maier
ec55a03ec1 more clippy 2021-04-07 19:53:14 +02:00
Dominik Maier
d72d48d6a8 more cargo fixes 2021-04-07 15:57:59 +02:00
Dominik Maier
8577d5c6a0 clippy fixes 2021-04-07 11:30:03 +02:00