464 Commits

Author SHA1 Message Date
julihoh
7231a66104
fix tutorial fuzzer (#323) 2021-10-10 16:13:01 +02:00
Andrea Fioraldi
a420eb0513
Qemu generic hooks (#313)
* generic hooks

* Fix generic hooks
2021-10-01 16:27:26 +02:00
Andrea Fioraldi
f2929b8253
Allowlist and denylist for QEMU edges and cmps (#311)
* fix edges hashtable query bug in hook

* fmt
2021-10-01 14:28:24 +02:00
Andrea Fioraldi
90928d3d97
Qemu Helpers and basic snapshotting (#310)
* store executor_ptr

* QemuHelpers

* working hooks and snapshot helper

* walk only the list of dirty pages on restore()

* mem hooks for snpashot

* brk snapshot

* snapshot method

* macos shit

* sugar and clippy
2021-10-01 12:17:28 +02:00
Toka
f63b862160
Frida for Windows (#287)
* harness.cc for win

* no backtrace for frida_gum

* build.rs message

* cfg guards

* at least libafl_frida builds with cfg guards

* fuzzer.rs builds on win

* clean up

* build instructions

* ps

* fix

* clang

* fix

* article

* static option to make it run on powershell

* vscode build instructions

* dllexport!

* fix

* build.rs

* fix & fmt

* message

* msys not necessary anymore

* Update README.md

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-09-29 22:10:15 +02:00
Dominik Maier
5a722994ac
Still fixing CI (#305)
* clippy

* clippy on windows

* clippy fixes on windows

* clippy, fmt

* fixed testcases for windows

* fixing workspace.yml

* testcase no longer fails without clang

* fix github dependencies to specific revisions

* fix qemu without python

* cleanup HookResult
2021-09-29 17:58:09 +02:00
Andrea Fioraldi
c6f7c3b3a8
Qemu new syscall hook and more python API (#306)
* new syscall hook

* expose more qemu to pylibafl

* hook syscalls from python

* update python example

* clippy

* clippy
2021-09-29 16:36:40 +02:00
Andrea Fioraldi
05aeb677cf
Fix default UBSan options and avoid timeouts in crash handler (#304)
* exit

* ignore timeouts outside the targets

* do not store timeouts

* block sigalarm in handlers

* __ubsan_default_options
2021-09-29 09:28:55 +02:00
Dominik Maier
2c51c4abf4 fmt 2021-09-28 21:01:35 +02:00
Dominik Maier
9d669bbc63
Fixed CI by ignoring python, resolved multiple warnings (#303)
* fixing ci

* ignoring dev deps

* fmt

* trying to fix dockerfile

* fix cargo build

* can't build sancov edges and hitcounts together

* fixed warnings

* fixed more warnings
2021-09-28 01:56:07 +02:00
Andrea Fioraldi
96bbe37cc1 Remove unused files from old libafl_qemu 2021-09-27 14:51:04 +02:00
Andrea Fioraldi
ed228147a0 Python QEMU example fuzzer 2021-09-27 14:48:44 +02:00
Andrea Fioraldi
40b73d4b1b
Book refactoring and update (#280)
* llmp docs skeleton

* llmp documentation

* more llmp docu

* llmp

* some core concepts

* start working on tutorial

* adapted rng_core to lain

* fix tutorial build

* warnings, format

* add explanation

* No need to own the types

* metadata

* writing

* fmt

* tutorial folder

* lain needs nightly

* added mdbook test to ci

* fix ci, add linkcheck

* more book

* baby

* tutorial target

* fix mdbook build

* fix mdbook test

* more book

* fixed typo

* fixed build

* spawn instances'

* 'finish' book

* added sugar crate information

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-09-27 09:39:32 +02:00
Dominik Maier
01a98bf8fd
Example how to build baby-fuzzer as push instead of pull, using Klo-routines (#227)
* iyielding fuzzer

* fixed klo example

* docu, naming

* more readme
2021-09-25 22:54:46 +02:00
Andrea Fioraldi
1fde608145
Qemu as lib (#301)
* linking problems

* use shared lib

* ci

* clippy, ci fixegit pushs

* ingoring distclean result

* clippy

* clippy

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-09-24 13:22:33 +02:00
Andrea Fioraldi
8f5df699fe fuzzbench_gsoc: set swarm num to 5 2021-09-24 13:20:50 +02:00
Andrea Fioraldi
fd39938ac7 Add fuzzbench gsoc to fuzzers/ 2021-09-24 11:23:26 +02:00
Dominik Maier
55c4b0c778
added write_file_atomic against ondisk corpus races (#294)
* fix ondisk corpus race condition

* move metadata name to be a dotfile

* note ExitKind for crashes and timeouts in inprocess executor

* potential fix for windows

* added write_file_atomic

* no_std fixes

* no_std testcase fix

* typo fix, windows

* clippy

* more no_std testing
2021-09-13 15:38:28 +02:00
Dominik Maier
9eba674467 implement Debug for EventManagerId 2021-09-12 12:26:08 +02:00
s1341
b4aa22458d
Add core_id to launcher run_client closure signature (#290)
* Add core_id to launcher run_client closure signature

* Format

* Attempt to fix windows build

* windows

* Sleep for index seconds instead of id seconds when launching cores (#292)
2021-09-12 09:49:36 +02:00
bitwave
e7ed5be9a2
Use external, custom time function for no_std environments (#281)
* Use external, custom time function for no_std environments

* fixup! Use external, custom time function for no_std environments

* fixup! Use external, custom time function for no_std environments
2021-09-06 19:13:45 +02:00
Andrea Fioraldi
14d1f63e56
Token level fuzzing (#274)
* EncodedInput

* some encoded mutations

* Encoded parse using comments and strings regexes

* working js example

* better mutator

* clippy
2021-08-27 09:32:22 +02:00
Andrea Fioraldi
6ae36ce658
Refactor configurations with EventConfig (#277)
* switch to EventConfig

* adapt fuzzers

* fix tests

* fix sugar

* clippy

* ahash
2021-08-26 13:25:03 +02:00
Andrea Fioraldi
7d249dd306 Update libafl-qemu-bridge commit && libafl_qemu 2021-08-24 16:03:33 +02:00
Andrea Fioraldi
951b6fef36 Bump 0.6.1 2021-08-23 10:01:22 +02:00
Toka
b6d22a2fbb
Drop for TimeoutExecutor on win10 (#269)
* small fixes

* update .gitignore
2021-08-19 09:11:16 +02:00
julihoh
dcf107a30a
symcc_runtime dependency fix for next libafl version & bump to 0.6.0 (#268)
* symcc_runtime dependency fix for next libafl version

* bump to 0.6.0

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-08-18 10:57:55 +02:00
Toka
d7ec395010
Forkserver Shared Memory Testcase (#265)
* working on shmem testcase fuzzing

* fmt & clippy

* write_to_testcase

* write input size

* max os fixes

* RcShMemProvider?

* ServedShMemProvider?

* revert changes

* RcShMem<ServedShMemProvider<MmapShMemProvider>>?

* ShMem change for android? (not tested at all)

* harness

* shmem testcase fuzzing for timeoutforkserver

* update harness

* remove .o

* pselect instead of select

* clippy
2021-08-17 01:44:17 +02:00
Toka
511237ce9e
Windows dependency update (#261)
* win018

* fuzzer fmt

* reorder

* comment

* does
2021-08-14 23:04:13 +02:00
Dominik Maier
ee33faa881
Bring back libpng CI on MacOS (#255)
* Makefile fix for libpng on MacOS (#252)

* clean exit for libpng_launcher
2021-08-09 02:59:39 +02:00
Dominik Maier
bb21ab7a63
Autolaunch ShMemService, add MacOS fuzzers to CI, various fixes (#246)
* starting to fix macos linker bugs

* mdetailed error prints

* start shmem service manually

* not a doc comment

* Some fixes

* only send exit msg to shmemservice when start was successful

* incorporated shmem service into provider

* removed unused imports

* trying to fix fuzzers

* fixed build

* check if join_handle is_some

* more debug prints

* fixed shmem service autolaunch

* fixed macos linker

* ignoring broken libpng testcase on macos for now (see #252)

* fmt

* try to fix missing llvm_config (see #253)

* empty issue template added

* Mmanually look for llvm-config on MacOS

* fixing CI

* fixed docs

* ignoring libmozjpg for CI, see #254
2021-08-09 01:10:29 +02:00
julihoh
e971f240da
Concolic example fuzzer fixes (#251)
* fix compilation of runtime of concolic example fuzzer

* fix compilation of example fuzzer

* fix incorrect traced target configuration

this would lead to the runtime never tracing any expressions.
failed to specifiy the input file name for the runtime to know what to symbolize

* add ability to specify whether a node should do concolic or traditional

* slightly more realistic concolic solving by using solver timeout

* enable expression pruning
2021-08-06 17:47:50 +02:00
julihoh
1dddf7e677
Separate symcc crate (#244)
* create a separate crate for symcc url and commit hash

also contains functions to checkout and build symcc from a build script

* fix dockerfile

* clippy
2021-08-05 21:57:39 +02:00
Dominik Maier
16c3a07be7
ShMem Server for MacOS (#238)
* generalized ashmem server

* fixed macos testcases

* added StdShMemService

* no_st

* fmt

* added testcase, fixed some bugs (not all)

* solidified unix shmem

* initial impl for MmapShMem

* Added shmem service start to more testcases

* clippy

* fixed tetcases

* added frida_libpng makefile for easy use

* trying to fix build on ubuntu

* fixed ubuntu build for libpng

* no_std

* fixed testcase
2021-08-05 17:08:01 +02:00
julihoh
704830a501
Documentation and Refactors for Concolic (#243)
* document symcc_runtime

* rename serialization format expressions to be more concise

* authorship notes

* document dump_constraints

* document smoke test

* tests for serialization format and refactoring

* remove unused bswap message

* remove obselete SymExpr::End

* document and refactor serialization format

* fmt

* more missing docs

* typos

* clippy
2021-08-05 15:01:14 +02:00
julihoh
3d98d31712
Concolic Tracing (#160)
* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* fixed build, clippy

* no_std

* implement WithObservers executor as discussed

* add symqemu as a submodule

* fix symqemu submodule URL to be relative

* update the concolic runtime to match the new interface

* update the trace file header regularly to save constraints in case the program crashes

* add build dependencies for symqemu

* handle full mesage buffer properly

* better policy for updating trace header

* less aggregiously inefficient GC information serialization

* move concolic runtime hitmap count to filter

this is in preparation for the new runtime interface

* very WIP new runtime interface

* use more convenient types in rust runtime

* EmptyRuntime -> NopRuntime

* hide cpp_runtime and formatting

* implement tracing runtime using new runtime interface

* implement filters with new runtime interface

* use a local checkout for symcc_runtime

* make test runtime tracing

* use test_runtime in smoke test

* fix formatting

* make the clippy overlord happy?

* disable symcc build on everything but linux

* make more of symcc_runtime linux only

* fix linking symcc_runtime with C++ stdlib

* will clippy ever be happy?

* formatting

* don't export symcc runtime when compiling tests

* clippy...

* "don't export symcc runtime when compiling tests" for runtime crate as well

* clippy

* move command executor to LibAFL

* move concolic crate into LibAFL

* move concolic{metada,observer} into LibAFL

* move ConcolicFeedback into LibAFL

* move ConolicStage into LibAFL

* fix bug in symcc part of concolic runtime

* stb_image fuzzer with concolic as example fuzzer

* clean up basic_concolic_fuzzer

* clean up and document concolic example fuzzer

* formatting

* clippy

* remove basic_concolic_fuzzer (it is now part of the examples)

* remove the runtime crate in favor of symcc_runtime

* re-architect concolic smoke test and remove git submodules

* remove old submodule directories

* make coverage filter public

* focker docker build

* clippy

* clippy fixes

* fix ubuntu as well

* remove .gitmodules

* move concolic mutational stage into libafl behind feature flag

* script to install dependencies for concolic smoke test

* fix bug

* clippy

* add github action to run smoke test

* fix action

* ensure smoke test is run in correct directory

* remove devcontainer files

* address feedback

* clippy

* more clippy

* address more feedback

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-08-05 13:22:00 +02:00
Dominik Maier
5542a81e12
Added state restorer testcase, fixed restorer (#240)
* added state restorer testcase

* fixed testcase

* no_std, clippy

* printing less often
2021-08-03 23:53:30 +02:00
Toka
bfe0a97077
AFLFast (#208)
* Defined PowerScheduleTestcase

* calibrate stage

* powerschedule

* PowerQueueCorpusScheduler

* Calstat

* Integer for observer

* update power.rs

* calculate_score

* no_std

* no_std

* calibrate_case

* calculate_score done

* update fuzz_level

* update depth

* rename, PowerScheduleQueueScheduler::next

* increment hashmap

* n_fuzz

* no_std

* fmt

* clippy

* check bitmap_size

* hash

* compile

* rename, vec

* fuzzer runs

* fixes

* rename, no_std log2

* fmt

* clippy

* fmt

* unused imports

* use exec_time

* getter/setter

* No unwrap

* ToString

* fixes

* ahash

* fmt

* use favored from power.rs side

* rename

* IsFavoredMetadata

* run_target

* clippy fixes

* doc & fix

* doc

* rename

* forgot log2

* fix

* fix

* added comments explaining why the COE and FAST schedules are different from the thesis

* saturated increment

* saturating_add

* put n_fuzz in PowerScheduleMetadata

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-28 21:19:50 +02:00
Andrea Fioraldi
2faf1d24c8
Hook syscalls in QemuBytesCoverageSugar (#233)
* add x64 syscalls numbers

* syscall hook

* update commit

* read guest mappings

* clippy

* read write hooks

* automerge fix

* type fix

* hooks syscalls in sugar
2021-07-21 12:29:46 +02:00
Andrea Fioraldi
db820d56a2
Qemu memory hooks (#232)
* add x64 syscalls numbers

* syscall hook

* update commit

* read guest mappings

* clippy

* read write hooks

* automerge fix

* type fix
2021-07-21 12:28:06 +02:00
Andrea Fioraldi
8e745f7d90 Remove useless fuzzbench_qemu build deps 2021-07-21 11:18:33 +02:00
Andrea Fioraldi
dfe39e2af7
libafl_sugar (#215)
* fuzzer mod

* libafl_sugar skeleton

* build libafl_sugar

* libfuzzer_stb_image_sugar

* Delete log

* qemu in libafl_sugar

* docker

* macos merda

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-07-20 11:47:33 +02:00
Andrea Fioraldi
5cd7339b1a
Qemu read memory mappings (#228)
* add x64 syscalls numbers

* syscall hook

* update commit

* read guest mappings

* clippy
2021-07-16 10:38:00 +02:00
Andrea Fioraldi
b09fa4e3f4
Qemu syscalls hook (#226)
* add x64 syscalls numbers

* syscall hook

* update commit
2021-07-15 11:12:10 +02:00
Andrea Fioraldi
42b7c6d7e7
Qemu CmpLog (#223)
* empty libafl_qemu crate

* fuzzbench qemu fuzzer skeleton

* emu.run() works without bp

* working emu loop

* resolve elf symbols

* running Qemu fuzzer without coverage

* qemu fuzzer with edge coverage

* merge into inprocess::GLOBAL_STATE

* create QemuExecutor and remove QemuEmulator

* qemu hooks and persist edges mapping storing them in State

* windows fix

* add libafl_qemu to workspace

* windows fix

* some clippy

* clippy

* fix fuzzbench_qemu

* fix fuzzbench_qemu makefile

* fuck you macos

* resolve PIC symbols

* cmp hooks

* cmplog hooks

* qemu cmplog

* clippy
2021-07-13 16:02:53 +02:00
Dominik Maier
712c5daeb9
Reload corpus size after restart (addresses #210) (#220)
* reload corpus size after restart (addresses #210)

* no_std
2021-07-12 13:16:40 +02:00
Dominik Maier
a0ba0f0251 clippy 2021-07-12 10:30:41 +02:00
Toka
4dea81b2a2
MOpt Refactor & Bug fixes (#218)
* rename

* fmt

* post_exec

* post_exec

* bug fix & change type

* refactor

* clippy

* fix

* unnecessary trait

* mode in Mutator

* remove println
2021-07-10 16:32:10 +02:00
Dominik Maier
aad271abf4
Cache Rust in CI (#217)
* rust cache

* less warnings during docker build

* removed unused use
2021-07-10 14:42:10 +02:00
Dominik Maier
0121096e84
Fixes for no_std build (#214)
* builds on no_std

* fixed std build

* nightly fmt on CI

* nightly fmt on CI (again)

* fmt

* no_std build on unix

* more mem

* added no_std from #212 to gh workflow

* more ci, less nightly

* clippy

* more toolchains?

* docu

* y u no build

* more ci?

* next try

* fixed dockr

* more dockerfile fixes

* ondisk corpus fixed

* panic:?

* ubunutu

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-09 20:07:56 +02:00