746 Commits

Author SHA1 Message Date
Dominik Maier
5caeb46b67
renamed target_os macos to target_vendor apple (#273)
* renamed target_os macos to target_vendor apple

* fix yaml
2021-08-23 09:45:25 +02:00
David CARLIER
8f03d20200
process crash handler, dump registers on apple arm64 (#271) 2021-08-21 14:47:55 +02:00
Toka
b6d22a2fbb
Drop for TimeoutExecutor on win10 (#269)
* small fixes

* update .gitignore
2021-08-19 09:11:16 +02:00
julihoh
dcf107a30a
symcc_runtime dependency fix for next libafl version & bump to 0.6.0 (#268)
* symcc_runtime dependency fix for next libafl version

* bump to 0.6.0

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-08-18 10:57:55 +02:00
Toka
688182fd1e
Timeout for Inprocess Executor on Windows (#267)
* start working on windows timeout

* salvage Input in timeout handler

* this time inproc_timeout_handler (need clean up later)

* cleaup

* more in inproc_timeout_handler

* fix for linux build

* more fixes for unix, fmt

* revert timeoutexecutor api

* revert baby_fuzzer/src/main.rs

* various fixes

* no unsafe

* remove timer in crash_handler
2021-08-18 09:11:34 +02:00
Toka
d7ec395010
Forkserver Shared Memory Testcase (#265)
* working on shmem testcase fuzzing

* fmt & clippy

* write_to_testcase

* write input size

* max os fixes

* RcShMemProvider?

* ServedShMemProvider?

* revert changes

* RcShMem<ServedShMemProvider<MmapShMemProvider>>?

* ShMem change for android? (not tested at all)

* harness

* shmem testcase fuzzing for timeoutforkserver

* update harness

* remove .o

* pselect instead of select

* clippy
2021-08-17 01:44:17 +02:00
Fabian Freyer
6435af3419
bolts::cpu: fix docs link (#264) 2021-08-15 07:08:37 +02:00
Toka
511237ce9e
Windows dependency update (#261)
* win018

* fuzzer fmt

* reorder

* comment

* does
2021-08-14 23:04:13 +02:00
David CARLIER
1418e836a0
shmem code simplification using more libc crate which fixes couple of wrong C fn signatures (#259) 2021-08-12 09:49:26 +02:00
Dominik Maier
cb8efd3abf
shmem service speedup env (#256)
* shmem service speedup env

* typos, cleanup

* fmt
2021-08-09 09:23:12 +02:00
Dominik Maier
bb21ab7a63
Autolaunch ShMemService, add MacOS fuzzers to CI, various fixes (#246)
* starting to fix macos linker bugs

* mdetailed error prints

* start shmem service manually

* not a doc comment

* Some fixes

* only send exit msg to shmemservice when start was successful

* incorporated shmem service into provider

* removed unused imports

* trying to fix fuzzers

* fixed build

* check if join_handle is_some

* more debug prints

* fixed shmem service autolaunch

* fixed macos linker

* ignoring broken libpng testcase on macos for now (see #252)

* fmt

* try to fix missing llvm_config (see #253)

* empty issue template added

* Mmanually look for llvm-config on MacOS

* fixing CI

* fixed docs

* ignoring libmozjpg for CI, see #254
2021-08-09 01:10:29 +02:00
Toka
7f4e341741
inprocessfork executor (#237)
* inprocessfork executor

* fmt

* cfg

* no_std

* no volatile rw

* wrapping_add

* fix

* mutable pointer

* ptr initialization in __sanitizer_cov_trace_pc_guard_init

* features

* more cfg

* fmt

* fix

* fmt

* post_fork

* fmt

* pre_fork

* test

* cfg
2021-08-07 12:09:54 +02:00
julihoh
e971f240da
Concolic example fuzzer fixes (#251)
* fix compilation of runtime of concolic example fuzzer

* fix compilation of example fuzzer

* fix incorrect traced target configuration

this would lead to the runtime never tracing any expressions.
failed to specifiy the input file name for the runtime to know what to symbolize

* add ability to specify whether a node should do concolic or traditional

* slightly more realistic concolic solving by using solver timeout

* enable expression pruning
2021-08-06 17:47:50 +02:00
Dominik Maier
16c3a07be7
ShMem Server for MacOS (#238)
* generalized ashmem server

* fixed macos testcases

* added StdShMemService

* no_st

* fmt

* added testcase, fixed some bugs (not all)

* solidified unix shmem

* initial impl for MmapShMem

* Added shmem service start to more testcases

* clippy

* fixed tetcases

* added frida_libpng makefile for easy use

* trying to fix build on ubuntu

* fixed ubuntu build for libpng

* no_std

* fixed testcase
2021-08-05 17:08:01 +02:00
julihoh
704830a501
Documentation and Refactors for Concolic (#243)
* document symcc_runtime

* rename serialization format expressions to be more concise

* authorship notes

* document dump_constraints

* document smoke test

* tests for serialization format and refactoring

* remove unused bswap message

* remove obselete SymExpr::End

* document and refactor serialization format

* fmt

* more missing docs

* typos

* clippy
2021-08-05 15:01:14 +02:00
julihoh
3d98d31712
Concolic Tracing (#160)
* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* fixed build, clippy

* no_std

* implement WithObservers executor as discussed

* add symqemu as a submodule

* fix symqemu submodule URL to be relative

* update the concolic runtime to match the new interface

* update the trace file header regularly to save constraints in case the program crashes

* add build dependencies for symqemu

* handle full mesage buffer properly

* better policy for updating trace header

* less aggregiously inefficient GC information serialization

* move concolic runtime hitmap count to filter

this is in preparation for the new runtime interface

* very WIP new runtime interface

* use more convenient types in rust runtime

* EmptyRuntime -> NopRuntime

* hide cpp_runtime and formatting

* implement tracing runtime using new runtime interface

* implement filters with new runtime interface

* use a local checkout for symcc_runtime

* make test runtime tracing

* use test_runtime in smoke test

* fix formatting

* make the clippy overlord happy?

* disable symcc build on everything but linux

* make more of symcc_runtime linux only

* fix linking symcc_runtime with C++ stdlib

* will clippy ever be happy?

* formatting

* don't export symcc runtime when compiling tests

* clippy...

* "don't export symcc runtime when compiling tests" for runtime crate as well

* clippy

* move command executor to LibAFL

* move concolic crate into LibAFL

* move concolic{metada,observer} into LibAFL

* move ConcolicFeedback into LibAFL

* move ConolicStage into LibAFL

* fix bug in symcc part of concolic runtime

* stb_image fuzzer with concolic as example fuzzer

* clean up basic_concolic_fuzzer

* clean up and document concolic example fuzzer

* formatting

* clippy

* remove basic_concolic_fuzzer (it is now part of the examples)

* remove the runtime crate in favor of symcc_runtime

* re-architect concolic smoke test and remove git submodules

* remove old submodule directories

* make coverage filter public

* focker docker build

* clippy

* clippy fixes

* fix ubuntu as well

* remove .gitmodules

* move concolic mutational stage into libafl behind feature flag

* script to install dependencies for concolic smoke test

* fix bug

* clippy

* add github action to run smoke test

* fix action

* ensure smoke test is run in correct directory

* remove devcontainer files

* address feedback

* clippy

* more clippy

* address more feedback

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-08-05 13:22:00 +02:00
Dominik Maier
4d50ba277a fixed typo 2021-08-04 15:29:42 +02:00
Dominik Maier
92ba3f59f9
StateRestorer.reset() removes old tmpfile (#242)
* StateRestorer.reset() removes old tmpfile

* checking map size on deref for extra safety

* clippy
2021-08-04 15:13:54 +02:00
Dominik Maier
5542a81e12
Added state restorer testcase, fixed restorer (#240)
* added state restorer testcase

* fixed testcase

* no_std, clippy

* printing less often
2021-08-03 23:53:30 +02:00
Dominik Maier
ff589d9a89 clippy fixes 2021-08-03 12:51:20 +02:00
julihoh
7750707fee
fix ci for latest clippy version (#239)
* clippy

* fix ubuntu as well
2021-08-03 12:29:30 +02:00
Dominik Maier
38c657338c added testcases for state restore, bugfixes 2021-08-03 05:12:51 +02:00
Toka
bfe0a97077
AFLFast (#208)
* Defined PowerScheduleTestcase

* calibrate stage

* powerschedule

* PowerQueueCorpusScheduler

* Calstat

* Integer for observer

* update power.rs

* calculate_score

* no_std

* no_std

* calibrate_case

* calculate_score done

* update fuzz_level

* update depth

* rename, PowerScheduleQueueScheduler::next

* increment hashmap

* n_fuzz

* no_std

* fmt

* clippy

* check bitmap_size

* hash

* compile

* rename, vec

* fuzzer runs

* fixes

* rename, no_std log2

* fmt

* clippy

* fmt

* unused imports

* use exec_time

* getter/setter

* No unwrap

* ToString

* fixes

* ahash

* fmt

* use favored from power.rs side

* rename

* IsFavoredMetadata

* run_target

* clippy fixes

* doc & fix

* doc

* rename

* forgot log2

* fix

* fix

* added comments explaining why the COE and FAST schedules are different from the thesis

* saturated increment

* saturating_add

* put n_fuzz in PowerScheduleMetadata

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-28 21:19:50 +02:00
Andrea Fioraldi
1692bbc214 Remove debug print 2021-07-28 15:12:28 +02:00
Andrea Fioraldi
0bd292f2ae Merge branch 'main' of github.com:AFLplusplus/LibAFL into main 2021-07-22 11:58:32 +02:00
Andrea Fioraldi
7e06b90364 Fix SimpleRestartingEventManager broken generics 2021-07-22 11:58:24 +02:00
s1341
5156b4cf8f
staterestore File::open should be File::create (#235) 2021-07-22 11:38:43 +02:00
David CARLIER
90b7ae08d6
x86 32 bits build fix (#234) 2021-07-21 21:48:33 +02:00
Andrea Fioraldi
dfe39e2af7
libafl_sugar (#215)
* fuzzer mod

* libafl_sugar skeleton

* build libafl_sugar

* libfuzzer_stb_image_sugar

* Delete log

* qemu in libafl_sugar

* docker

* macos merda

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-07-20 11:47:33 +02:00
Dominik Maier
9591ed995e
Panic on Too Slow Broker (#230)
* panic on too slow broker

* constant for max pending pages
2021-07-20 02:39:03 +02:00
David CARLIER
b0cb74324c
fix android arm64 build (#231) 2021-07-20 02:38:44 +02:00
Andrea Fioraldi
230d7a1cb1
CachedOnDiskCorpus (#229)
* sized inputs

* push CachedOnDiskCorpus and fix OnDiskCorpus

* comments
2021-07-16 16:34:43 +02:00
Andrea Fioraldi
5cd7339b1a
Qemu read memory mappings (#228)
* add x64 syscalls numbers

* syscall hook

* update commit

* read guest mappings

* clippy
2021-07-16 10:38:00 +02:00
Dominik Maier
5a14b870e2
Added staterestore to restarting mgrs (#225)
* added staterestore to simple restarting mgr

* reworked launcher

* ? instead of unwrap

* no_std fixes

* windows

* fixed save fn

* added llvm to dockerfile
2021-07-15 13:13:07 +02:00
Andrea Fioraldi
b09fa4e3f4
Qemu syscalls hook (#226)
* add x64 syscalls numbers

* syscall hook

* update commit
2021-07-15 11:12:10 +02:00
Andrea Fioraldi
ac783c6119 Fix inifnite serialization for CmpValuesMetadata 2021-07-15 09:58:23 +02:00
Andrea Fioraldi
ec4c5ae88a
remove timeout on crash (#224) 2021-07-13 16:15:12 +02:00
Andrea Fioraldi
42b7c6d7e7
Qemu CmpLog (#223)
* empty libafl_qemu crate

* fuzzbench qemu fuzzer skeleton

* emu.run() works without bp

* working emu loop

* resolve elf symbols

* running Qemu fuzzer without coverage

* qemu fuzzer with edge coverage

* merge into inprocess::GLOBAL_STATE

* create QemuExecutor and remove QemuEmulator

* qemu hooks and persist edges mapping storing them in State

* windows fix

* add libafl_qemu to workspace

* windows fix

* some clippy

* clippy

* fix fuzzbench_qemu

* fix fuzzbench_qemu makefile

* fuck you macos

* resolve PIC symbols

* cmp hooks

* cmplog hooks

* qemu cmplog

* clippy
2021-07-13 16:02:53 +02:00
Dominik Maier
712c5daeb9
Reload corpus size after restart (addresses #210) (#220)
* reload corpus size after restart (addresses #210)

* no_std
2021-07-12 13:16:40 +02:00
Toka
13b4a143b8
branch main (#219) 2021-07-11 01:47:01 +02:00
Toka
4dea81b2a2
MOpt Refactor & Bug fixes (#218)
* rename

* fmt

* post_exec

* post_exec

* bug fix & change type

* refactor

* clippy

* fix

* unnecessary trait

* mode in Mutator

* remove println
2021-07-10 16:32:10 +02:00
Dominik Maier
0121096e84
Fixes for no_std build (#214)
* builds on no_std

* fixed std build

* nightly fmt on CI

* nightly fmt on CI (again)

* fmt

* no_std build on unix

* more mem

* added no_std from #212 to gh workflow

* more ci, less nightly

* clippy

* more toolchains?

* docu

* y u no build

* more ci?

* next try

* fixed dockr

* more dockerfile fixes

* ondisk corpus fixed

* panic:?

* ubunutu

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-09 20:07:56 +02:00
Andrea Fioraldi
4af9af784f Serialize CmpValuesMetadata to an empty object to save space on State serialize 2021-07-09 14:35:31 +02:00
Andrea Fioraldi
d472a1242a
libafl_qemu (#211)
* empty libafl_qemu crate

* fuzzbench qemu fuzzer skeleton

* emu.run() works without bp

* working emu loop

* resolve elf symbols

* running Qemu fuzzer without coverage

* qemu fuzzer with edge coverage

* merge into inprocess::GLOBAL_STATE

* create QemuExecutor and remove QemuEmulator

* qemu hooks and persist edges mapping storing them in State

* windows fix

* add libafl_qemu to workspace

* windows fix

* some clippy

* clippy

* fix fuzzbench_qemu

* fix fuzzbench_qemu makefile

* fuck you macos
2021-07-08 15:21:14 +02:00
Andrea Fioraldi
a1fc2a5453 Bump to 0.5.0 2021-07-05 14:12:40 +02:00
Toka
849ff1fa04
MOpt scheduler (#161)
* add the struct for MOpt globals

* constants

* RAND_C

* more comments & reorder class members

* select_algorithm

* no_std fixes

* clippy fixes

* MOptMutator

* MutatorsTuple has HasLen

* MOptStage

* pso_update

* HasMOpt trait

* ScheduledMutator, core_fuzzing

* clippy fix

* fmt

* core_fuzzing

* core_fuzzing done

* fix

* pilot_mutate

* pilot_fuzzing

* pilot_fuzzing done

* MOpt metadata

* Make MOptMutator into a trait

* initialize_mopt

* No getter/setters

* fmt

* fixed compiler warnings & clippy warnings

* Comments

* fix type paramter, integrate into libpng

* fmt

* fmt

* No HasMOpt

* fmt

* improve

* pso_initialize, various fixes

* clippy

* fmt

* always pacemaker mode

* fmt

* fix

* less noisy fmt::Debug

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-07-05 13:54:15 +02:00
Andrea Fioraldi
5b76c22ea7
Cmplog routines mutator (#204)
* save

* routines in meta

* execute passes

* fix cmplog rtn pass

* clippy
2021-07-05 09:54:44 +02:00
Dominik Maier
84a9e36acf
Github Actions Cache (#205)
* trying out github actions cache

* split up build steps

* deactivated cache for macos

* debugging

* also for windows (?))

* clippy fix

* out folder ignored
2021-07-02 20:51:22 +02:00
Andrea Fioraldi
f0743cbb17
remove serde bounds for Observer (#202) 2021-07-02 15:07:51 +02:00
Andrea Fioraldi
44f6e4c389
Improve introspection (#200)
* remove NUM_FEEDBACKS

* working introspection

* adust introspection stats

* bugfixes, clippy

* removed outdated define

* more clippy;

* no_std

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-07-02 10:58:36 +02:00