42 Commits

Author SHA1 Message Date
Dongjia "toka" Zhang
6f21cb3848
Bump to 0.10.1 (#1280)
* bmp

* remove DEBUG env var
2023-05-22 12:42:36 +02:00
Dominik Maier
96e24d1c8b
Move Input loading and dumping APIs from Testcase to Corpus (#1201)
* Less allocatiosn for filenames

* clippy for wasm fuzzer

* Reworked filename and rename APIs

* python, clippy

* fmt

* More cleanup, fixed metadata location

* clippy

* fix fuzzbench_text / cached len, invert parameters (state first)

* clippy

* oops

* Caching for paths

* simplified, fixed

* no_std

* cached_len

* Nider API for input getting
2023-04-18 12:14:49 +02:00
Andrea Fioraldi
0f633962ff
Bump to 0.10.0 (#1156)
* Bump to 0.10.0

* fix

* Fix CI

* Fix copyright

* fmt

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-04-05 17:49:53 +02:00
Dominik Maier
f4f23de32b
Fix fuzzers after HasTestcase (#1123) (#1162)
* Fix fuzzers after HasTestcase (#1123)

* Make the trait a trait

* Implement HasTestcase for Corpora

* fix

* fix

* a

* a

* fix

* wasm32

* a

* f

* f

* aa

---------

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2023-03-19 11:58:32 +09:00
Dominik Maier
bbe4e85768
Removed new_ from constructors that don't need it (API consistency) (#1159)
* Removed new_ from constructors that don't need it (API consistency)

* un-change python bindings
2023-03-17 17:02:21 +01:00
Matheus Baptistella
c38405ef83
Shorthand functions to get typed metadata, renamed metatdata -> metadata_map (#1123)
* Created macro to get the metadata form State and Testcase

* Expanded the macros for mutable, or not, State and Testcase metadata

* Created functions on traits HasMetadata and HasNamedMetadatato get, mutable or not, metadata

* Created the functions to get metadata

* Added #[inline] attribute and renamed the functions

* Renamed the functions and added #[inline] attribute

* Temporarily added testcase() function

* Added testcase() function

* Changed Ref import to core::cell:Ref

* Added testcase_mut() and renamed occurences of metadata() and metadata_mut()

* Renamed more occurences

* Renamed the metadata() on impl HasMetadata for NopState

---------

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2023-03-14 14:24:33 +01:00
Andrea Fioraldi
dc800f0814
on_evaluation Scheduler method (#1106)
* add on evaluation hook in schedulers

* on_evaluation for WeightedScheduler

* fix PowerQueueScheduler

* fix fuzzers

* upd qemu

* tests

* upd
2023-02-28 11:33:26 +01:00
Addison Crump
e42cd9c12f
Fixes for on_replace/on_remove and related for StdFuzzer and MapFeedback (#1067)
* scheduler replace fixes

* oops, no-std

* add

* changes on the fuzzers

* move map feedback history updates to append_metadata

* fixes for python bindings

* learn to clippy

* fix for fuzzer add_input

* clippy fixes for frida

* additional powersched differences

* corrections for bitmap_size

* off-by-one

* I live in a prison of my own creation and clippy is the warden

* clear the novelties map for the situation where is_interesting is invoked, but not append_metadata

---------

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2023-02-15 17:04:18 +01:00
Dongjia "toka" Zhang
30b51bb810
Remove unused dependencies (#1069) 2023-02-14 09:02:18 +01:00
R. Elliott Childre
5d76707ede
Bump deps and fix Clippy warns in example fuzzers (#1043)
* Mostly addressing changing the `uninlined_format_args` lint which was
  changed to warn-by-default in rust clippy 1.67

* Bump dependencies:
  bindgen:  0.61 -> 0.63
  cc:       1.0 -> 1.0.42 (Exclue versions w/incompat rayon dependency)
  clap:     3.x -> 4.0
  rangemap: 0.1 -> 1
  xz -> xz2:  move to updated version

* Add fallthrough default return to `LLVMFuzzerTestOneInput` in
  **/fuzz.c to prevent Clang's -Wreturn-type

* libafl_atheris: Improve POSIX compatibility and reduce warnings
  * Check for .dylib and .so libraries
  * `source` -> `.` for POSIX shells
  * install wheel into the venv to support newer Python packaging
    standards
  * `LDPRELOAD` -> `LD_PRELOAD`
2023-02-05 21:53:45 +01:00
Andrea Fioraldi
eaf5ff9de0
Restart loading initial inputs even after a crash/timeout (#1040)
* Track initial inputs loading

* libfuzzer libpng

* fuzzbench

* fix no_std

* fix no_std

* clippy

* fuzzers
2023-02-03 11:56:47 +01:00
Andrea Fioraldi
fdf579bcd5
Bump to 0.9.0 (#946)
* bump to 0.9.0

* fix libafl_tinyinst

* fix

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-01-31 10:42:01 +01:00
Dominik Maier
97e88af0c5
OnDiskCorpus: Write metadata by default, metadata gzip compression (#995)
* Write metadata by default

* fix fuzzers

* Cleanup, gzip feature

* Fix casing for ondisk corpus

* fix fmt, clippy

* clippy

* clippy for gdiplus fuzzer

* fmt
2023-01-13 01:07:36 +01:00
Addison Crump
28786c943a
Grimoire fixes (#993)
* fixup grimoire/generalisation, remove GeneralizedInput in favour of metadata

* additional cleanup

* transformable inputs to solve the grimoire problem

* explicit use of 'transforming' to keep typing compatible with normal usage

* clippy fix

* fixes for nautilus, python

* explicit inlining for reflexive impl

* fix for tutorial

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2023-01-13 01:07:21 +01:00
Dominik Maier
75f12bd0eb
Remodelling Observers/Examples that rely on UB, API cleanups (#950)
* Tackling UB

* PtrMut -> MutPtr, moved mapobservers to non-UB

* QEMU fixes

* test fixes

* qemu

* Change all interfaces, fix all fuzzers

* fixes

* fix more fixes

* fmt

* fix qemu sugar

* fix some qemus

* atheris

* fmt

* more fmt

* most fmt

* more fix

* nyx fyx

* fix qemu

* clippy, fixes

* more fixes

* no unfix, only fix

* fix

* fix

* more clippy

* fixes

* ListObserver

* fmt, clippy

* fix qemu on arm

* update zlib target

* fix?

* fix

* added migration guide

* ignore doc

* fix symcc

* fix new win fuzzer

* Fixes, rename PTR_SIZE to PTR_NUM

* Try fix linking on win

* Trying to fix win linking

* more cov

* trying to fix win some more

* trying to fix mac

* trying to fix mac

* Fix tests

* Fix tests

* trying to fix win

* more mac

* giving up for windows

* fmt

* python3

* mac?

* undo windows tests
2022-12-24 14:20:44 +01:00
Dominik Maier
663a33168e
Associated types for Corpus, State (#767)
* Associated types for Corpus, State

* cleanup

* fix no_std

* drop unused clauses

* Corpus

* cleanup

* adding things

* fixed fuzzer

* remove phantom data

* python

* progress?

* more more

* oof

* wow it builds?

* python fixes, tests

* fix python fun

* black fmt for python

* clippy, added Nop things

* fixes

* fix merge

* make it compile (#836)

* doc-test fixes, prelude-b-gone for cargo-hack compat

* fixes for windows, concolic

* really fix windows, maybe

* imagine using windows

* ...

* elide I generic when used with S: State

* Elide many, many generics, but at what cost?

* progress on push

* Constraint HasCorpus, HasSolutions at trait definition

* remove unused feature

* remove unstable usage since we constrained HasCorpus at definition

* compiled, but still no type inference for MaxMapFeedback

* cleanup inprocess

* resolve some std conflicts

* simplify map

* undo unnecessary cfg specification

* fix breaking test case for CI on no-std

* fix concolic build failures

* fix macos build

* fixes for windows build

* timeout fixes for windows build

* fix pybindings issues

* fixup qemu

* fix outstanding local build issues

* maybe fix windows inprocess

* doc fixes

* unbridled fury

* de-associate State from Feedback, replace with generic as AT inference is not sufficient to derive specialisation for MapFeedback

* merge update

* refactor + speed up fuzzer builds by sharing build work

* cleanup lingering compiler errors

* lol missed one

* revert QEMU-Nyx change, not sure how I did that

* move HasInput to inputs

* HasInput => KnowsInput

* update bounds to enforce via associated types

* disentangle observers with fuzzer

* revert --target; update some fuzzers to match new API

* resolve outstanding fuzzer build blockers (that I can run on my system)

* fixes for non-linux unixes

* fix for windows

* Knows => Uses, final fixes for windows

* <guttural screaming>

* fixes for concolic

* loosen bound for frida executor so windows builds correctly

* cleanup generics for eventmanager/eventprocessor to drop observers requirement

* improve inference over fuzz_one and friends

* update migration notes

* fixes for python bindings

* fixes for generic counts in event managers

* finish migration notes

* post-merge fix

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2022-10-24 03:22:26 +02:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
Dongjia Zhang
eac7307c5a
0.8.1 (#732) 2022-08-18 10:23:57 +02:00
syheliel
2504b6dae3
Add rustfmt.toml (#722)
* add `rustfmt.toml`

* format fix

Co-authored-by: syheliel <syheliel@gmail.com>
2022-08-12 02:28:32 +02:00
Dominik Maier
12052b5f1c
Remove num_cpus dependency (#717)
* Remove num_cpus dependency

* Fix build, remove more num_cpus
2022-08-03 07:49:32 +02:00
s1341
c45b6be7e1
Bump rand version (#680)
* Bump rand version, dropping support for lain

* using lain version with fixed rand

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-07-16 01:46:32 +02:00
Dongjia Zhang
ffe8dbf6af
Bump to 0.8.0 (#696)
* upd

* more
2022-07-11 21:59:11 +02:00
Andrea Fioraldi
da537aae83
FeedbackState as metadata (#627)
* SerdeAny MapFeedbackState

* Fix macro syntax

* alloc

* fix

* Metadata calibrate and map feedback

* metadata feedback states

* compile

* fmt

* Register common generic types

* tests

* sugar

* no_std

* fix book

* alloc

* fix fuzzers

* fix

* fmt

* disable python bindings for libafl

* clippy

* fmt

* fixes

* fmt

* fix

* fix

* fix

* fix

* fix

* release autofix

* fix

* fix

* fix

* fmt

* fix

* fix

* name

* fix

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-05-24 16:05:22 +02:00
Dongjia Zhang
283ceaac9b
Make weigthed scheduler independent of powersheduler stage (#599)
* rename & add metadata in scheduler, not stage

* Update testcase_score

* rename

* fix

* update handicap in scheduler

* fmt

* update fuzzers

* doc

* fmt

* fix

* fmt

* more

* fix

* fix

* fix

* fmt
2022-05-08 16:43:02 +02:00
Toka
abf1a66028
Rename FavFactor to TestcaseScore; More TestcaseScores (#574)
* rework aflfast

* more

* move fuzz_Mu

* weighted

* fix

* borrow checker fix

* compute_weight

* alias_table

* fmt

* fix & rename

* fix & less mut

* no_std

* no_std

* clippy

* 32bit clippy fix

* top_rated for compute_weight

* fix

* clippy & metadata Init

* fix

* fix

* fix

* clippy & fmt

* change fuzzers

* fuzzbench_selected

* fmt

* compute() has state

* use favfactor for powerschedules also

* fix merge

* rename

* fmt & clippy

* no_std

* fmt

* clippy

* rename

* fmt

* rename

* fmt

* fix

* fix

* fmt

* fix

* fix
2022-03-27 04:04:46 +09:00
Dongjia Zhang
c72f773ca0
Weighted corpus entry selection (#570)
* rework aflfast

* more

* move fuzz_Mu

* weighted

* fix

* borrow checker fix

* compute_weight

* alias_table

* fmt

* fix & rename

* fix & less mut

* no_std

* no_std

* clippy

* 32bit clippy fix

* top_rated for compute_weight

* fix

* clippy & metadata Init

* fix

* fix

* fix

* clippy & fmt

* change fuzzers

* fuzzbench_selected

* fmt
2022-03-23 02:01:00 +09:00
Andrea Fioraldi
a56f4af7da
CorpusScheduler -> Scheduler and move them to the schedulers folder (#560)
* CorpusScheduler -> Scheduler

* fix book

* update fuzzers

* fix tests

* fix sugar

* fix

* fix tutorial

* fix tutorial

* fmt

* fix

* fmt

* fmt
2022-03-03 14:27:37 +01:00
Youssef
e307dfb16f
Implement backtrace observers for crash dedupe (#379)
* create stacktrace observer

* create stacktrace feedback

* post-merge fixes

* address comments

* update Cargo.toml

* fix CI issue + dynamic naming

* duplicate baby_fizzer

* update stacktrace baby_fuzzer

* force unwinding tables

* ignore test dumps

* fix stacktrace baby_fuzzer logic

* upgrade Backtrace version

* trigger observers.post_exec in crash_handler

* implement NewHashFeedbackState and update logic

* digest symbols pointers

* cleanup

* minimal output

* fix backdated EventFirer generic param

* add baby_fuzzer example with a fork executor

* duplicate baby_fuzzer_stacktrace with forkexecutor

* backtrace collection implemented

* add c app fuzzer example with a fork executor

* group backtrace baby fuzzers

* added c code baby fuzzer with inprocess executor

* remove need for static COLLECT_BACKTRACE

* moved code to stacktrace.rs + fixed bug

* add comment

* add command executor fuzzer example

* post merge cleanup

* add missing doc

* address comment

* fix nit

* clean duplicate variable in timeout handler

* fix command executor bt collection

* clean code and use StdShMem

* cleanup

* add ObserverWithHashField + rename StacktraceObserver

* rename + refactor some code

* add CommandBacktraceObserver

* update command executor

* update baby fuzzers

* simplify BacktraceSharedMemoryWrapper

* use better names + static methods

* use std feature macro on BacktraceObserver + fix bug

* use Box in HashValueWrapper to minimize variants size diff

* use copy_from_slice

* std conditional backtrace collection

* fix std import

* fix comment

* add exit_kind to observer.post_exec

* added hash trait to Input

* collect backtrace in post_exec

* add crash handlers to InProcessForkExecutor

* fix panic message

* duplicate forkserver fuzzer example

minimal example

update

* proto bt collection working

* rename CommandBacktraceExecutor to ASANBacktraceExecutor

* refactor ASANBacktraceObserver

* support for forkserver working

* update fuzzer example

* less verbosity

* Post merge fixes

* implement hash for GeneralizedInput

* update forkserver example after merge

* clippy fixes

* fix inproc test

* fixes for cargo hack --feature-powerset

* fix baby_no_std

* implement Hash for NautilusInput

* update fork executor baby fuzzer

* fix doc

* implement Hash for PacketData

* fix windows build

* fix windows no_std

* fix backtrace baby fuzzers README

* add comments

* move setup_bt_panic to constructor

* pre/post child exec hooks in Observer

* setup_child_panic_hook

* fix ObserversOwnedMap on nightly

* add backtrace fuzzers to CI checks

* fix typo

* fix relative paths in test_all_fuzzers.sh

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-01-31 15:58:15 +01:00
Dominik Maier
77e5965e97
Add AsSlice, AsMutSlice traits, refactor MapObservers to be iterable, and have associated types (#477)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic

* refacctor map observers

* iterator for map observers

* removed unused ownedptr, added asslice trait to most functions

* make map entry type an associated type

* fix fuzzers

* fix docs

* typo fix

* fix windows, add try_from_slice to shmid

* missing import

* fix fuzzbench

* cleanup

* fmt

* more asslice

* fmt

* added doc link about token-level fuzzing

* cods
2022-01-19 00:02:33 +01:00
Dominik Maier
2dd88998bd
Clippy fixes for latest toolchain (#471)
* from warning

* fix latest clippy

* clippy fixes++

* more nits
2022-01-17 11:02:42 +01:00
Andrea Fioraldi
bca1f392a7
Bump to 0.7.1 (#465)
* bump to 0.7.1

* bump libafl_qemu
2022-01-13 11:32:57 +01:00
Evan Richter
250ec8d1e0
Reduce generics for various Has* traits (#456)
Specifically for Has{Rand,Corpus,Solutions,FeedbackStates}

The Has* family of traits offer getters and get-mut-ers. The previous
implementation had a fully generic return type:

    trait HasX<X: TraitX> {
        get_x(&self) -> &Self::X;
        get_mut_x(&mut self) -> &mut Self::X;
    }

meaning a single type could implement both `HasRand<Romu>` and
`HasRand<XorShift>`. The advantage of having multiple implementations is
not clear at this time, so it vastly simplifies the trait (and its
impls) to bring the return type in the body as an associated type:

    trait HasX {
        type X: TraitX;
        get_x(&self) -> &Self::X;
        get_mut_x(&mut self) -> &mut Self::X;
    }

This comes with the limitation that any type that impls these traits can
only do so once, choosing only one associated type.

* HasRand's only generic parameter (Rand) is now an associated type
* HasCorpus and HasSolutions are now only generic over the Input type
  they store
* HasFeedbackStates generic parameter now associated type
2022-01-06 10:41:02 +01:00
Dominik Maier
30eb1508de
Add OwnedSlice::RefRaw to keep track of raw pointers (#448)
* add OwnedSlice::RefRaw to keep track of raw pointers

* clippy

* fmt

* new from ownedref

* clippy

* OwnedSliceInner

* fix,From

* as_slice()

* fmt

* fix doc

* OwnedSliceMut

* fixes

* clippy

* fix

* ownedmut -> owned

* to owned

* to_owned -> clone

* removed comment

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2022-01-05 01:15:23 +01:00
Dominik Maier
af3d321213
Derive debug for all structs in LibAFL (#442)
* documentation, warnings

* fixed docs

* docs

* no_std

* test

* windows

* nautilus docs

* more fixes

* more docs

* nits

* windows clippy

* docs, windows

* nits

* debug all the things

* derive debug for all core library components

* Docu for libafl_targets

* nits

* reordered generics

* add docs to frida, debug

* nits

* fixes

* more docu for frida, nits

* more docu

* more docu

* Sugar docs

* debug for qemu

* more debug

* import debug

* fmt

* debug

* anyap_debug feature no longer needed

* tidy up unused fn

* indicate if we left out values for struct debug

* implement Debug for sugar

* debug allthethings

* ci
2022-01-03 00:47:17 +01:00
Andrea Fioraldi
37b8cb0d2f Bump to 0.7 2021-12-01 17:22:09 +01:00
Dominik Maier
ba969108e3
Push stage trait (#380)
* rpush mutational trait

* tiny changes

* started PushStageAdapter

* fmt

* refactoring

* fix docs

* no_std

* formatted more
2021-11-17 12:51:14 +01:00
Toka
23f02dae12
Fix api (#376)
* Fix api

* change fuzzers
2021-11-12 19:10:53 +01:00
Dominik Maier
62afed61e2
Renamed Stats to Monitors (#373)
* renamed stats to monitors

* added depreciation notices

* resorted generics alphaabetically

* added monitors

* fmt fuzzers

* added depreciation note for usermonitor

* fmt all fuzzers script

* more fmt

* renamed some monitor things back to stats

* fixed rename
2021-11-12 11:01:08 +01:00
Andrea Fioraldi
eca605bf01
MultiMapObserver and sancov 8bit-counters instrumentation (#343)
* MultiMapObserver and 8bit instrumentation

* fix test

* clippy

* fix

* fix tutorial

* sancov_8bit targets feature
2021-11-04 10:08:50 +01:00
Jordan McLeod
3d436b7519
Upgrade to Rust 2021 Edition (#340)
* Enable missing const_xxh3 feature

* Move to Rust 2021 Edition

* Fix clippy complaints

* Remove imports made unecessary in 2021 edition
2021-11-04 09:59:49 +01:00
julihoh
7231a66104
fix tutorial fuzzer (#323) 2021-10-10 16:13:01 +02:00
Andrea Fioraldi
40b73d4b1b
Book refactoring and update (#280)
* llmp docs skeleton

* llmp documentation

* more llmp docu

* llmp

* some core concepts

* start working on tutorial

* adapted rng_core to lain

* fix tutorial build

* warnings, format

* add explanation

* No need to own the types

* metadata

* writing

* fmt

* tutorial folder

* lain needs nightly

* added mdbook test to ci

* fix ci, add linkcheck

* more book

* baby

* tutorial target

* fix mdbook build

* fix mdbook test

* more book

* fixed typo

* fixed build

* spawn instances'

* 'finish' book

* added sugar crate information

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-09-27 09:39:32 +02:00