580 Commits

Author SHA1 Message Date
Andrea Fioraldi
23edffd4c1 Ooops, fmt 2021-10-18 10:57:33 +02:00
Andrea Fioraldi
0ed9dc6d80 Panic message in the map feedback 2021-10-18 10:56:39 +02:00
Andrea Fioraldi
bc4770fb82 Revert "Error message in most likely case of using NONASAN and ASAN fuzzers using the same Fuzzer config (#329)"
This reverts commit d71f0cf5bf4fc449585a4087e5fcf4a8eb443e38.
2021-10-18 10:46:08 +02:00
Marcin Kozlowski
d71f0cf5bf
Error message in most likely case of using NONASAN and ASAN fuzzers using the same Fuzzer config (#329)
* Error message in most likely case of using NONASAN and ASAN fuzzers using the same Fuzzer config

* Typo

* Changed to panic. Executed cargo fmt
2021-10-15 09:56:26 +02:00
Andrea Fioraldi
a782e8faaa CmpValuesMetadata empty serialization 2021-10-14 17:04:03 +02:00
Andrea Fioraldi
279fb3f213 Launcher: restore fs::File but open just before dup2 2021-10-14 16:10:07 +02:00
Andrea Fioraldi
ab7672c8d1 Launcher: use libc::open instead of fs::File 2021-10-14 15:57:32 +02:00
julihoh
2e55d24f5a
update deps (#327)
* experimental update deps

* Reverted to rand-core 0.5.1 for Lain compatibility

* updated nix

* less strict libc dep

* remove deprecated errno conversion

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-10-12 10:32:21 +02:00
julihoh
7930d42948
update packages related to concolic (#325)
* update packages related to concolic

* install z3 on CI macOS

I have no clue how this worked before...

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-10-11 21:24:37 +02:00
julihoh
6e2aa47285
add ability to trace location information in concolic tracer (#322)
* add ability to trace location information in concolic tracer

* fix formatting

* introduce location new-type

* fix conolic smoke test

* impl From instead of custom into_inner function

* fmt

* change to use usize instead of NonZeroUsize

in order to no over-constrain the implementation
2021-10-11 13:31:16 +02:00
David CARLIER
939784d512
dumping process address maps on netbsd (#316) 2021-10-06 00:18:40 +02:00
Andrea Fioraldi
90928d3d97
Qemu Helpers and basic snapshotting (#310)
* store executor_ptr

* QemuHelpers

* working hooks and snapshot helper

* walk only the list of dirty pages on restore()

* mem hooks for snpashot

* brk snapshot

* snapshot method

* macos shit

* sugar and clippy
2021-10-01 12:17:28 +02:00
Dominik Maier
5a722994ac
Still fixing CI (#305)
* clippy

* clippy on windows

* clippy fixes on windows

* clippy, fmt

* fixed testcases for windows

* fixing workspace.yml

* testcase no longer fails without clang

* fix github dependencies to specific revisions

* fix qemu without python

* cleanup HookResult
2021-09-29 17:58:09 +02:00
Andrea Fioraldi
c6f7c3b3a8
Qemu new syscall hook and more python API (#306)
* new syscall hook

* expose more qemu to pylibafl

* hook syscalls from python

* update python example

* clippy

* clippy
2021-09-29 16:36:40 +02:00
Andrea Fioraldi
05aeb677cf
Fix default UBSan options and avoid timeouts in crash handler (#304)
* exit

* ignore timeouts outside the targets

* do not store timeouts

* block sigalarm in handlers

* __ubsan_default_options
2021-09-29 09:28:55 +02:00
Dominik Maier
9d669bbc63
Fixed CI by ignoring python, resolved multiple warnings (#303)
* fixing ci

* ignoring dev deps

* fmt

* trying to fix dockerfile

* fix cargo build

* can't build sancov edges and hitcounts together

* fixed warnings

* fixed more warnings
2021-09-28 01:56:07 +02:00
Andrea Fioraldi
40b73d4b1b
Book refactoring and update (#280)
* llmp docs skeleton

* llmp documentation

* more llmp docu

* llmp

* some core concepts

* start working on tutorial

* adapted rng_core to lain

* fix tutorial build

* warnings, format

* add explanation

* No need to own the types

* metadata

* writing

* fmt

* tutorial folder

* lain needs nightly

* added mdbook test to ci

* fix ci, add linkcheck

* more book

* baby

* tutorial target

* fix mdbook build

* fix mdbook test

* more book

* fixed typo

* fixed build

* spawn instances'

* 'finish' book

* added sugar crate information

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-09-27 09:39:32 +02:00
Lukas Seidel
f0d5c2f708
armv7 support: add ucontext struct definition (#297)
* add uncontext and mcontext struct defs for armv7

* formatting

* merge import lines
2021-09-14 13:58:03 +02:00
Dominik Maier
55c4b0c778
added write_file_atomic against ondisk corpus races (#294)
* fix ondisk corpus race condition

* move metadata name to be a dotfile

* note ExitKind for crashes and timeouts in inprocess executor

* potential fix for windows

* added write_file_atomic

* no_std fixes

* no_std testcase fix

* typo fix, windows

* clippy

* more no_std testing
2021-09-13 15:38:28 +02:00
Toka
b9edb29d8b
PowerSchedule::COE fix (#295) 2021-09-13 14:24:31 +02:00
Dominik Maier
9eba674467 implement Debug for EventManagerId 2021-09-12 12:26:08 +02:00
s1341
b4aa22458d
Add core_id to launcher run_client closure signature (#290)
* Add core_id to launcher run_client closure signature

* Format

* Attempt to fix windows build

* windows

* Sleep for index seconds instead of id seconds when launching cores (#292)
2021-09-12 09:49:36 +02:00
Dominik Maier
b71704b14d
ShMem server race-condition fix for #276 (#278)
* This fixes a potential race condition when the parent dies before the child connects after a fork (#276)

* fix docs

* trying to fix shmem server forking

* removed bug where decreasing map count to 0 would not be reallocatable

* ignored clippy warning, refactoring
2021-09-07 00:03:37 +02:00
bitwave
e7ed5be9a2
Use external, custom time function for no_std environments (#281)
* Use external, custom time function for no_std environments

* fixup! Use external, custom time function for no_std environments

* fixup! Use external, custom time function for no_std environments
2021-09-06 19:13:45 +02:00
Dominik Maier
d8ef1dd90a fix Android build on MacOS 2021-09-06 12:53:58 +02:00
Dominik Maier
9cd34865a8 removed unused import 2021-09-06 12:07:21 +02:00
Andrea Fioraldi
42d213737d
Build id configuration in std (#286)
* Build id configuration in std

* uuid only on std
2021-09-06 10:25:32 +02:00
Andrea Fioraldi
231caf0797 reset() method for FeedbackState 2021-09-06 10:00:26 +02:00
Toka
d136ee7427
Fix _LLMP_BIND_ADDR for Windows (#285)
* ipv6

* client connects to localhost

* v4 when v6 not available

* remove v6 addr
2021-09-03 12:36:49 +02:00
David CARLIER
774cfb685e
openbsd port (#279)
with libc update, ucontext data is finally available on this platform too.
2021-08-27 11:41:06 +02:00
Andrea Fioraldi
14d1f63e56
Token level fuzzing (#274)
* EncodedInput

* some encoded mutations

* Encoded parse using comments and strings regexes

* working js example

* better mutator

* clippy
2021-08-27 09:32:22 +02:00
Andrea Fioraldi
6ae36ce658
Refactor configurations with EventConfig (#277)
* switch to EventConfig

* adapt fuzzers

* fix tests

* fix sugar

* clippy

* ahash
2021-08-26 13:25:03 +02:00
Andrea Fioraldi
951b6fef36 Bump 0.6.1 2021-08-23 10:01:22 +02:00
s1341
fccb9a1c55
Fix shmem on android (#272)
* Fix shmem on android

* Switch back to ulong
2021-08-23 09:46:54 +02:00
Dominik Maier
5caeb46b67
renamed target_os macos to target_vendor apple (#273)
* renamed target_os macos to target_vendor apple

* fix yaml
2021-08-23 09:45:25 +02:00
David CARLIER
8f03d20200
process crash handler, dump registers on apple arm64 (#271) 2021-08-21 14:47:55 +02:00
Toka
b6d22a2fbb
Drop for TimeoutExecutor on win10 (#269)
* small fixes

* update .gitignore
2021-08-19 09:11:16 +02:00
julihoh
dcf107a30a
symcc_runtime dependency fix for next libafl version & bump to 0.6.0 (#268)
* symcc_runtime dependency fix for next libafl version

* bump to 0.6.0

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-08-18 10:57:55 +02:00
Toka
688182fd1e
Timeout for Inprocess Executor on Windows (#267)
* start working on windows timeout

* salvage Input in timeout handler

* this time inproc_timeout_handler (need clean up later)

* cleaup

* more in inproc_timeout_handler

* fix for linux build

* more fixes for unix, fmt

* revert timeoutexecutor api

* revert baby_fuzzer/src/main.rs

* various fixes

* no unsafe

* remove timer in crash_handler
2021-08-18 09:11:34 +02:00
Toka
d7ec395010
Forkserver Shared Memory Testcase (#265)
* working on shmem testcase fuzzing

* fmt & clippy

* write_to_testcase

* write input size

* max os fixes

* RcShMemProvider?

* ServedShMemProvider?

* revert changes

* RcShMem<ServedShMemProvider<MmapShMemProvider>>?

* ShMem change for android? (not tested at all)

* harness

* shmem testcase fuzzing for timeoutforkserver

* update harness

* remove .o

* pselect instead of select

* clippy
2021-08-17 01:44:17 +02:00
Fabian Freyer
6435af3419
bolts::cpu: fix docs link (#264) 2021-08-15 07:08:37 +02:00
Toka
511237ce9e
Windows dependency update (#261)
* win018

* fuzzer fmt

* reorder

* comment

* does
2021-08-14 23:04:13 +02:00
David CARLIER
1418e836a0
shmem code simplification using more libc crate which fixes couple of wrong C fn signatures (#259) 2021-08-12 09:49:26 +02:00
Dominik Maier
cb8efd3abf
shmem service speedup env (#256)
* shmem service speedup env

* typos, cleanup

* fmt
2021-08-09 09:23:12 +02:00
Dominik Maier
bb21ab7a63
Autolaunch ShMemService, add MacOS fuzzers to CI, various fixes (#246)
* starting to fix macos linker bugs

* mdetailed error prints

* start shmem service manually

* not a doc comment

* Some fixes

* only send exit msg to shmemservice when start was successful

* incorporated shmem service into provider

* removed unused imports

* trying to fix fuzzers

* fixed build

* check if join_handle is_some

* more debug prints

* fixed shmem service autolaunch

* fixed macos linker

* ignoring broken libpng testcase on macos for now (see #252)

* fmt

* try to fix missing llvm_config (see #253)

* empty issue template added

* Mmanually look for llvm-config on MacOS

* fixing CI

* fixed docs

* ignoring libmozjpg for CI, see #254
2021-08-09 01:10:29 +02:00
Toka
7f4e341741
inprocessfork executor (#237)
* inprocessfork executor

* fmt

* cfg

* no_std

* no volatile rw

* wrapping_add

* fix

* mutable pointer

* ptr initialization in __sanitizer_cov_trace_pc_guard_init

* features

* more cfg

* fmt

* fix

* fmt

* post_fork

* fmt

* pre_fork

* test

* cfg
2021-08-07 12:09:54 +02:00
julihoh
e971f240da
Concolic example fuzzer fixes (#251)
* fix compilation of runtime of concolic example fuzzer

* fix compilation of example fuzzer

* fix incorrect traced target configuration

this would lead to the runtime never tracing any expressions.
failed to specifiy the input file name for the runtime to know what to symbolize

* add ability to specify whether a node should do concolic or traditional

* slightly more realistic concolic solving by using solver timeout

* enable expression pruning
2021-08-06 17:47:50 +02:00
Dominik Maier
16c3a07be7
ShMem Server for MacOS (#238)
* generalized ashmem server

* fixed macos testcases

* added StdShMemService

* no_st

* fmt

* added testcase, fixed some bugs (not all)

* solidified unix shmem

* initial impl for MmapShMem

* Added shmem service start to more testcases

* clippy

* fixed tetcases

* added frida_libpng makefile for easy use

* trying to fix build on ubuntu

* fixed ubuntu build for libpng

* no_std

* fixed testcase
2021-08-05 17:08:01 +02:00
julihoh
704830a501
Documentation and Refactors for Concolic (#243)
* document symcc_runtime

* rename serialization format expressions to be more concise

* authorship notes

* document dump_constraints

* document smoke test

* tests for serialization format and refactoring

* remove unused bswap message

* remove obselete SymExpr::End

* document and refactor serialization format

* fmt

* more missing docs

* typos

* clippy
2021-08-05 15:01:14 +02:00
julihoh
3d98d31712
Concolic Tracing (#160)
* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* fixed build, clippy

* no_std

* implement WithObservers executor as discussed

* add symqemu as a submodule

* fix symqemu submodule URL to be relative

* update the concolic runtime to match the new interface

* update the trace file header regularly to save constraints in case the program crashes

* add build dependencies for symqemu

* handle full mesage buffer properly

* better policy for updating trace header

* less aggregiously inefficient GC information serialization

* move concolic runtime hitmap count to filter

this is in preparation for the new runtime interface

* very WIP new runtime interface

* use more convenient types in rust runtime

* EmptyRuntime -> NopRuntime

* hide cpp_runtime and formatting

* implement tracing runtime using new runtime interface

* implement filters with new runtime interface

* use a local checkout for symcc_runtime

* make test runtime tracing

* use test_runtime in smoke test

* fix formatting

* make the clippy overlord happy?

* disable symcc build on everything but linux

* make more of symcc_runtime linux only

* fix linking symcc_runtime with C++ stdlib

* will clippy ever be happy?

* formatting

* don't export symcc runtime when compiling tests

* clippy...

* "don't export symcc runtime when compiling tests" for runtime crate as well

* clippy

* move command executor to LibAFL

* move concolic crate into LibAFL

* move concolic{metada,observer} into LibAFL

* move ConcolicFeedback into LibAFL

* move ConolicStage into LibAFL

* fix bug in symcc part of concolic runtime

* stb_image fuzzer with concolic as example fuzzer

* clean up basic_concolic_fuzzer

* clean up and document concolic example fuzzer

* formatting

* clippy

* remove basic_concolic_fuzzer (it is now part of the examples)

* remove the runtime crate in favor of symcc_runtime

* re-architect concolic smoke test and remove git submodules

* remove old submodule directories

* make coverage filter public

* focker docker build

* clippy

* clippy fixes

* fix ubuntu as well

* remove .gitmodules

* move concolic mutational stage into libafl behind feature flag

* script to install dependencies for concolic smoke test

* fix bug

* clippy

* add github action to run smoke test

* fix action

* ensure smoke test is run in correct directory

* remove devcontainer files

* address feedback

* clippy

* more clippy

* address more feedback

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-08-05 13:22:00 +02:00