48 Commits

Author SHA1 Message Date
Andrea Fioraldi
7b0039606b
Forksrv adaptive map size and AFL++ CmpLog support (#896)
* AFL++ cmplog map

* map size opt in forkserver

* MapObserver::downsize_map and adaptive map size in forkserver

* fix fokserver_simple cmd opts

* clippy

* fuzzbench forkserver with cmplog

* delete makefile in fuzzbench forkserver

* fuzzbench_forkserver is persistent

* ForkserverExecutorBuilder::build_dynamic_map

* fix

* clippy

* fix

* fix macos

* fix compilation

* fix bugs

* fixes

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-22 10:33:15 +01:00
Lukas Seidel
17a0d9e8f0
Forkserver: Add file input support (#880)
* make use of clap derive in forkserver_simple

* (re)introduce use_shmem_testcase flag to ForkserverExecutor

* set use_shmem_testcase flag automatically based on forkserver handshake

* remove illegal_state and just .unwrap instead as the None case is unreachable

* fix: removed pub method

* cargo fmt

* remove illegal_state #2 and just .unwrap instead as the None case is unreachable

* change shmem unwrap to unwrap_unchecked

* fix double mut

* removed @@ warning
2022-11-10 15:25:52 +01:00
Dominik Maier
663a33168e
Associated types for Corpus, State (#767)
* Associated types for Corpus, State

* cleanup

* fix no_std

* drop unused clauses

* Corpus

* cleanup

* adding things

* fixed fuzzer

* remove phantom data

* python

* progress?

* more more

* oof

* wow it builds?

* python fixes, tests

* fix python fun

* black fmt for python

* clippy, added Nop things

* fixes

* fix merge

* make it compile (#836)

* doc-test fixes, prelude-b-gone for cargo-hack compat

* fixes for windows, concolic

* really fix windows, maybe

* imagine using windows

* ...

* elide I generic when used with S: State

* Elide many, many generics, but at what cost?

* progress on push

* Constraint HasCorpus, HasSolutions at trait definition

* remove unused feature

* remove unstable usage since we constrained HasCorpus at definition

* compiled, but still no type inference for MaxMapFeedback

* cleanup inprocess

* resolve some std conflicts

* simplify map

* undo unnecessary cfg specification

* fix breaking test case for CI on no-std

* fix concolic build failures

* fix macos build

* fixes for windows build

* timeout fixes for windows build

* fix pybindings issues

* fixup qemu

* fix outstanding local build issues

* maybe fix windows inprocess

* doc fixes

* unbridled fury

* de-associate State from Feedback, replace with generic as AT inference is not sufficient to derive specialisation for MapFeedback

* merge update

* refactor + speed up fuzzer builds by sharing build work

* cleanup lingering compiler errors

* lol missed one

* revert QEMU-Nyx change, not sure how I did that

* move HasInput to inputs

* HasInput => KnowsInput

* update bounds to enforce via associated types

* disentangle observers with fuzzer

* revert --target; update some fuzzers to match new API

* resolve outstanding fuzzer build blockers (that I can run on my system)

* fixes for non-linux unixes

* fix for windows

* Knows => Uses, final fixes for windows

* <guttural screaming>

* fixes for concolic

* loosen bound for frida executor so windows builds correctly

* cleanup generics for eventmanager/eventprocessor to drop observers requirement

* improve inference over fuzz_one and friends

* update migration notes

* fixes for python bindings

* fixes for generic counts in event managers

* finish migration notes

* post-merge fix

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2022-10-24 03:22:26 +02:00
Dominik Maier
e8b3d33bf4
Update dependencies, removed unused deps, CI fixes (#839)
* update clap, remove unused deps

* update grammartek

* update pyo3

* update pyo3

* undid clap update

* not changing nyx

* updated deps

* Update more deps, fixes

* not needed clippy

* fix windows

* try to enable deprecated pyproto for pyo3

* unused

* moving some things to clap4 after all

* initial move to clap 4

* fix clap

* more clap4, removed accidental file

* fixes, fmt

* fix

* all fix no play

* fix
2022-10-18 20:36:43 +02:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
Dongjia Zhang
eac7307c5a
0.8.1 (#732) 2022-08-18 10:23:57 +02:00
syheliel
2504b6dae3
Add rustfmt.toml (#722)
* add `rustfmt.toml`

* format fix

Co-authored-by: syheliel <syheliel@gmail.com>
2022-08-12 02:28:32 +02:00
Dominik Maier
5a8bdae26f
Update requirements (#714)
* Update requirements

* more updates

* nits

* more updates

* update nix for fuzzers

* use any regex 1 instead of 1.6
2022-07-29 09:54:16 +02:00
Dongjia Zhang
ffe8dbf6af
Bump to 0.8.0 (#696)
* upd

* more
2022-07-11 21:59:11 +02:00
Dongjia Zhang
c9f802a3b8
Improve map feedback/observer (#665)
* improve

* a

* fix it back

* more

* NO

* try

* fix python

* more

* specialize map map feedback with u8

* more

* fmt

* usable_count = len

* clp

* restore iterator based map feedback

* simd specialization

* optimize hitcounts

* fix hitcounts

* no_std

* moar unsafe

* fix

* clippy

* clippy

* opt non-specialized is_interesting

* fmt

* op post_exec

* cleanup

* even more

* allow

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-06-10 10:14:12 +02:00
Dominik Maier
bfe69aea09
Format C/Cpp code in ./scripts/fmt_all.sh (#653)
* format all (clang format 13)

* added clang-format check

* re-add missing newline

* cargo doc was missing

* more brackets

* fixed fmt workflow

* clang format

* shellcheck

* install clang-format-13

* update ubuntu for maximum clang-formattability

* yml whitespaces

* fmt

* shellcheck only for .sh

* oops path

* ignored shellcheck warning
2022-05-29 03:23:02 +02:00
Andrea Fioraldi
da537aae83
FeedbackState as metadata (#627)
* SerdeAny MapFeedbackState

* Fix macro syntax

* alloc

* fix

* Metadata calibrate and map feedback

* metadata feedback states

* compile

* fmt

* Register common generic types

* tests

* sugar

* no_std

* fix book

* alloc

* fix fuzzers

* fix

* fmt

* disable python bindings for libafl

* clippy

* fmt

* fixes

* fmt

* fix

* fix

* fix

* fix

* fix

* release autofix

* fix

* fix

* fix

* fmt

* fix

* fix

* name

* fix

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-05-24 16:05:22 +02:00
WilliamParks
92196cc9be
Fixes forkersever_simple issue on Macs (#623)
* Fixes forkersever_simple issue on Macs

* fixes formatting issue

* Fixes formatting issue
2022-05-08 11:17:55 +09:00
Dongjia Zhang
2ba32c0173
Update Clap dependency (#621) 2022-05-06 01:12:25 +02:00
Andrea Fioraldi
a56f4af7da
CorpusScheduler -> Scheduler and move them to the schedulers folder (#560)
* CorpusScheduler -> Scheduler

* fix book

* update fuzzers

* fix tests

* fix sugar

* fix

* fix tutorial

* fix tutorial

* fmt

* fix

* fmt

* fmt
2022-03-03 14:27:37 +01:00
Tamas K Lengyel
b3d68e8f40
Add signal option to forkserver_simple (#548) 2022-02-21 16:49:04 +01:00
Dongjia Zhang
936e2221d1
Cargo-make (#537)
* timeout utility

* example build.toml

* upd

* ci

* Update build_and_test.yml

* Update build_and_test.yml

* rename, qemu_launcher

* libpngs

* fix

* upd

* del

* do_nothing -> unsupported

* rename

* use command

* non qemu fuzzbench

* script.sh

* mroe

* qemu

* fix

* generic

* fix

* fix

* allow 124

* quotes

* fix

* fix

* fix

* stderr to devnull

* chg
2022-02-20 03:32:43 +01:00
Dongjia Zhang
42cab49f3e
Forkserver builder fix (#529)
* fix

* fix

* fmt

* no @@

* fuzzer change

* parse_afl_cmdline

* comma
2022-02-11 09:38:26 +01:00
Dongjia Zhang
9d38fff662
Autodict forkserver (#525)
* Builder for ForkserverExecutor

* add

* clippy warnings

* comment

* stash

* tmp

* change

* revert

* use_shmem_feature field

* change the harness back

* wip

* wip

* revert

* works

* clippy

* Makefile fix

* doc

* clippy

* rename to program

* rename, fix, envs

* lifetime

* arg_input_file

* stash

* read autodict from forkserver

* works

* clippy & fmt

* fmt

* fix

* fix

* fmt

* better harness

* arg_input_file_std

* rename

* fix
2022-02-10 10:27:51 +01:00
Dongjia Zhang
9482433e54
Forkserver builder (#523)
* Builder for ForkserverExecutor

* add

* clippy warnings

* comment

* stash

* tmp

* change

* revert

* use_shmem_feature field

* change the harness back

* wip

* wip

* revert

* works

* clippy

* Makefile fix

* doc

* clippy

* rename to program

* rename, fix, envs

* lifetime

* arg_input_file

* bug fix

* arg_input_file

* builder()

* doc

* clippy & fmt

* clippy & fmt
2022-02-09 22:07:15 +01:00
Sagittarius-a
14959c7f9c
Fix debug_child arg in forkserver_simple example (#489)
The `debug_child` command line argument presence was not properly checked,
so it couldn't be set to true. Hence it was not possible to print out
the content of the buffer sent to the harness while fuzzing.
2022-01-22 09:42:05 +01:00
Dominik Maier
77e5965e97
Add AsSlice, AsMutSlice traits, refactor MapObservers to be iterable, and have associated types (#477)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic

* refacctor map observers

* iterator for map observers

* removed unused ownedptr, added asslice trait to most functions

* make map entry type an associated type

* fix fuzzers

* fix docs

* typo fix

* fix windows, add try_from_slice to shmid

* missing import

* fix fuzzbench

* cleanup

* fmt

* more asslice

* fmt

* added doc link about token-level fuzzing

* cods
2022-01-19 00:02:33 +01:00
Dominik Maier
4f6f76e857
Streamline ShMem API (#472)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic
2022-01-17 18:28:26 +01:00
Andrea Fioraldi
bca1f392a7
Bump to 0.7.1 (#465)
* bump to 0.7.1

* bump libafl_qemu
2022-01-13 11:32:57 +01:00
Dominik Maier
6d9763c51f
Move to clap 3.0 (#447)
* move to clap 3.0

* fix cargo.toml

* update symcc to use clap3
2022-01-04 23:53:12 +01:00
Dominik Maier
af3d321213
Derive debug for all structs in LibAFL (#442)
* documentation, warnings

* fixed docs

* docs

* no_std

* test

* windows

* nautilus docs

* more fixes

* more docs

* nits

* windows clippy

* docs, windows

* nits

* debug all the things

* derive debug for all core library components

* Docu for libafl_targets

* nits

* reordered generics

* add docs to frida, debug

* nits

* fixes

* more docu for frida, nits

* more docu

* more docu

* Sugar docs

* debug for qemu

* more debug

* import debug

* fmt

* debug

* anyap_debug feature no longer needed

* tidy up unused fn

* indicate if we left out values for struct debug

* implement Debug for sugar

* debug allthethings

* ci
2022-01-03 00:47:17 +01:00
Dominik Maier
217a7dee1d
Use Structopt instead of yaml for example fuzzers, introduce Cores API (#420)
* reworked generic_inmemory to structopt

* moved core parsing to a struct

* added Cores

* added structopt to libpng_ctx

* improved libafl, added structopt to libpng launcher

* fix deexit ub

* move more to structopt

* improve llvm-config detection

* move construct_automata to structopt

* clippy, fixes, ...

* no_std

* clippy

* frida core parsing

* fixed no-fork cores

* updated clap

* added missing import

* missing borrow

* reworked frida to structopt

* fixed build

* using Cores api for atheris

Co-authored-by: Dominik Maier <d.maier@avm.de>
2021-12-15 03:58:35 +01:00
Dongjia Zhang
a96e01fda5
Fix forkserver_simple clap issue (#412) 2021-12-10 03:38:42 +01:00
Dongjia Zhang
bfcb86a2e4
Forkserver example updated (#404) 2021-12-06 13:48:41 +01:00
Dongjia Zhang
30f8fd44ef
Better forkserver example (#399)
* better example

* fmt
2021-12-04 16:17:38 +01:00
Andrea Fioraldi
37b8cb0d2f Bump to 0.7 2021-12-01 17:22:09 +01:00
Dominik Maier
62afed61e2
Renamed Stats to Monitors (#373)
* renamed stats to monitors

* added depreciation notices

* resorted generics alphaabetically

* added monitors

* fmt fuzzers

* added depreciation note for usermonitor

* fmt all fuzzers script

* more fmt

* renamed some monitor things back to stats

* fixed rename
2021-11-12 11:01:08 +01:00
Jordan McLeod
3d436b7519
Upgrade to Rust 2021 Edition (#340)
* Enable missing const_xxh3 feature

* Move to Rust 2021 Edition

* Fix clippy complaints

* Remove imports made unecessary in 2021 edition
2021-11-04 09:59:49 +01:00
Toka
c0d40a43e1
remove libafl_tests (#324)
* remove libafl_tests

* fmt

* fix

* fix

* fix
2021-10-10 20:44:49 +02:00
Andrea Fioraldi
951b6fef36 Bump 0.6.1 2021-08-23 10:01:22 +02:00
julihoh
dcf107a30a
symcc_runtime dependency fix for next libafl version & bump to 0.6.0 (#268)
* symcc_runtime dependency fix for next libafl version

* bump to 0.6.0

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-08-18 10:57:55 +02:00
Toka
d7ec395010
Forkserver Shared Memory Testcase (#265)
* working on shmem testcase fuzzing

* fmt & clippy

* write_to_testcase

* write input size

* max os fixes

* RcShMemProvider?

* ServedShMemProvider?

* revert changes

* RcShMem<ServedShMemProvider<MmapShMemProvider>>?

* ShMem change for android? (not tested at all)

* harness

* shmem testcase fuzzing for timeoutforkserver

* update harness

* remove .o

* pselect instead of select

* clippy
2021-08-17 01:44:17 +02:00
Dominik Maier
bb21ab7a63
Autolaunch ShMemService, add MacOS fuzzers to CI, various fixes (#246)
* starting to fix macos linker bugs

* mdetailed error prints

* start shmem service manually

* not a doc comment

* Some fixes

* only send exit msg to shmemservice when start was successful

* incorporated shmem service into provider

* removed unused imports

* trying to fix fuzzers

* fixed build

* check if join_handle is_some

* more debug prints

* fixed shmem service autolaunch

* fixed macos linker

* ignoring broken libpng testcase on macos for now (see #252)

* fmt

* try to fix missing llvm_config (see #253)

* empty issue template added

* Mmanually look for llvm-config on MacOS

* fixing CI

* fixed docs

* ignoring libmozjpg for CI, see #254
2021-08-09 01:10:29 +02:00
Dominik Maier
16c3a07be7
ShMem Server for MacOS (#238)
* generalized ashmem server

* fixed macos testcases

* added StdShMemService

* no_st

* fmt

* added testcase, fixed some bugs (not all)

* solidified unix shmem

* initial impl for MmapShMem

* Added shmem service start to more testcases

* clippy

* fixed tetcases

* added frida_libpng makefile for easy use

* trying to fix build on ubuntu

* fixed ubuntu build for libpng

* no_std

* fixed testcase
2021-08-05 17:08:01 +02:00
Dominik Maier
aad271abf4
Cache Rust in CI (#217)
* rust cache

* less warnings during docker build

* removed unused use
2021-07-10 14:42:10 +02:00
Andrea Fioraldi
a1fc2a5453 Bump to 0.5.0 2021-07-05 14:12:40 +02:00
Toka
fbeec3ca6c
Faster feedback (#206)
* faster_feedback

* typo

* feedback_or

* comma

* crash dedup can use the fast one
2021-07-05 13:51:18 +02:00
Toka
c01f1e3318
launcher example needs --cores (#203) 2021-07-02 16:52:27 +02:00
Toka
b3c52a4ad6
Test fuzzers (#187)
* build_all_fuzzer.shj

* run.sh

* output log

* ENABLE_SHARED off

* libc6-dev

* echo

* no need to cargo build twice

* replaced realpath (not available on macos) with /Users/domenukk/tmp/libaflrs/fuzzers/libfuzzer_libpng_launcher

* replaced PWD with pwd

* trying to get llvm-config working

* more sudo?

* slash

* trying to get all deps

* more info

* delete apt install from build_all_fuzzers.sh

* correct libfuzzer_libpng makefile

* fix build for libfuzzer_libpng

* fix other makefiles

* nproc not supported on macos

* no run.sh, use make short_test

* enable_shared=false

* just Linux

* fix

* forkserver makefile

* fix

* stb_image Makefile

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-02 15:35:41 +02:00
Andrea Fioraldi
bdb5efbf5b
Configurations (#162)
* print sender id

* storing sender id to env

* executor in llmp handle_in_client

* compile the lib

* compiling generic_inmemory

* fix forkserver

* adapt from fuzzers

* instrospection fix

* exitkind in NewTestcase

* fix libafl_frida

* fix firda_libpng

* send conf with Newtestcase event

* bump to 0.4.0

* no_std fix

* fmt

* fix libfuzzer_libmozjpeg

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-06-22 15:04:14 +02:00
Toka
e4b3cc542a
Forkserver_simple uses TimeoutForkserverExecutor (#139)
* forkserver_simple uses TimeoutforkserverExecutor

* fmt

* from_millis
2021-05-31 15:44:56 +02:00
Dominik Maier
945693f6ed fixed example testcase 2021-05-26 16:50:10 +02:00
Toka
d4410c072a
ForkserverExecutor (#111)
* add Forkserver, Pipe Outfile struct

* add forkserver executor struct, and shmem init

* close pipes in the destructor of Forkserver

* fill pre_exec to write out the inputs

* fix

* read_st, write_ctl

* more handshakes

* wrap Pipe in Arc, fill post_exec

* add Forkserver, Pipe Outfile struct

* add forkserver executor struct, and shmem init

* close pipes in the destructor of Forkserver

* fill pre_exec to write out the inputs

* fix

* read_st, write_ctl

* more handshakes

* wrap Pipe in Arc, fill post_exec

* fix for the lastest HasExecHooks trait

* use Dominik's pipe, remove Arc and temporarily pass RawFd to setstdin but trying to figure out other solutions

* add libafl_tests, put a very simple vulnerable program

* fix

* added forkserver_simple (mostly copy-pasted from babyfuzzer)

* fix test

* handle crash in post_exec

* add README.md

* check exec time to see why it's so slow

* remove double invokation of is_interesting for the obejctive

* make forkserver_simple AFL-like and improve speed

* some debugging help

* do not evaluate feedback if solution

* speedup the things

* working input placement via stdin in Forkserver

* don't call panic! but return errors, rewrite some comments

* use AFLplusplus/afl-cc instead of AFL

* use .cur_input like AFL

* bring the test for forkserver back

* add better README.md message

* failing the initial handshake should return an error

* delete some commented-out code

* format

* format

* ForkserverExecutor needs std and is unix-only for now

* clippy

* OutFile error handling

* fmt

* clippy

* don't build libafl_tests on windows

* fix

* keep test in forkserver.rs simple

* add forkserver_test feature for libafl_tests

* format

* some doc

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-05-25 13:40:00 +02:00