41 Commits

Author SHA1 Message Date
Dominik Maier
68fbfc8914
Fix Clippy (#926)
* Fix clippy

* undo comment fmt

* add unstracked nyx files to gitignore

* fix

* windows, no_std

* fix

* fix

* more

* macos

* remove doctest

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2022-12-06 00:05:42 +09:00
julihoh
948c94d695
Update and fix concolic support (#901)
* fix incorrect assert condition and document it

* update symcc

* adapt to changes in symcc API

* more fixes

* fix formatting

* more fixes

* speed up smoke test by building multiple crates in one command

* update symcc commit to latest main
2022-11-19 23:05:15 +01:00
Dominik Maier
e8b3d33bf4
Update dependencies, removed unused deps, CI fixes (#839)
* update clap, remove unused deps

* update grammartek

* update pyo3

* update pyo3

* undid clap update

* not changing nyx

* updated deps

* Update more deps, fixes

* not needed clippy

* fix windows

* try to enable deprecated pyproto for pyo3

* unused

* moving some things to clap4 after all

* initial move to clap 4

* fix clap

* more clap4, removed accidental file

* fixes, fmt

* fix

* all fix no play

* fix
2022-10-18 20:36:43 +02:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
Dominik Maier
94f0c7f56e
Moving to named parameters in format strings (#827)
* autofix

* you're just asking for a clamping

* autofmt on linux

* fix nits

* change back nit

* unfixing as u64 for GuestAddr

* fix

* ignoring clippy for GuestAddress
2022-10-11 13:45:01 +02:00
Dominik Maier
5823320206
Fixes for new Clippy (#755) 2022-09-02 16:21:06 +02:00
Dongjia Zhang
eac7307c5a
0.8.1 (#732) 2022-08-18 10:23:57 +02:00
syheliel
2504b6dae3
Add rustfmt.toml (#722)
* add `rustfmt.toml`

* format fix

Co-authored-by: syheliel <syheliel@gmail.com>
2022-08-12 02:28:32 +02:00
Dominik Maier
5a8bdae26f
Update requirements (#714)
* Update requirements

* more updates

* nits

* more updates

* update nix for fuzzers

* use any regex 1 instead of 1.6
2022-07-29 09:54:16 +02:00
Dongjia Zhang
3c0c95e382
upd (#697) 2022-07-12 10:31:56 +02:00
Dongjia Zhang
ffe8dbf6af
Bump to 0.8.0 (#696)
* upd

* more
2022-07-11 21:59:11 +02:00
Dominik Maier
e4447364c2
SymCC update (#656) 2022-05-28 01:41:43 +02:00
Andrea Fioraldi
da537aae83
FeedbackState as metadata (#627)
* SerdeAny MapFeedbackState

* Fix macro syntax

* alloc

* fix

* Metadata calibrate and map feedback

* metadata feedback states

* compile

* fmt

* Register common generic types

* tests

* sugar

* no_std

* fix book

* alloc

* fix fuzzers

* fix

* fmt

* disable python bindings for libafl

* clippy

* fmt

* fixes

* fmt

* fix

* fix

* fix

* fix

* fix

* release autofix

* fix

* fix

* fix

* fmt

* fix

* fix

* name

* fix

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-05-24 16:05:22 +02:00
Dongjia Zhang
2ba32c0173
Update Clap dependency (#621) 2022-05-06 01:12:25 +02:00
Dominik Maier
5df130188a
Fixing CI from #559 (#580)
* updated ref

* update symcc

* updated symcc

* CI for symcc

* updated symcc

* enabling git

* add runtime deps to makefile

* only linux

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2022-03-29 14:57:38 +02:00
Andrea Fioraldi
a56f4af7da
CorpusScheduler -> Scheduler and move them to the schedulers folder (#560)
* CorpusScheduler -> Scheduler

* fix book

* update fuzzers

* fix tests

* fix sugar

* fix

* fix tutorial

* fix tutorial

* fmt

* fix

* fmt

* fmt
2022-03-03 14:27:37 +01:00
Dominik Maier
7dad2153e2
Clippy for Cargo (#532)
* Clippy for Cargo

* clippy fixes

* clippy fixes

* edition

* fix

* wrong self hidden

* fix

* more clippy
2022-02-11 14:34:01 +01:00
Dominik Maier
6810e6085b
Builder for CommandExecutor & Tokens Refactoring (#508)
* builder for CommandExecutor

* tokens api cleanup, clippy

* fix doctest

* cleanup

* added testcase, remodelled

* command executor builder fix

* fix fuzzer(?)

* implemented From for configurator

* nits

* clippy

* unused

* autotokens

* cleanup

* nits

* Err instead of empty tokens

* fix tokens fn

* fix err

* more error fixing

* tokens remodelling

* typo

* recoverable fail on missing autotokens

* clippy, nostd

* asslice, into_iter, etc. for tokens

* adapt fuzzers

* iter

* fixes, clippy

* fix

* more clippy

* no_std

* more fix

* fixed typo

* cmd_executor builds again

* bring back ASAN stuff to Command Executor

* forkserver speedup

* no need to static

* back to earlier
2022-02-01 10:10:47 +01:00
Dominik Maier
77e5965e97
Add AsSlice, AsMutSlice traits, refactor MapObservers to be iterable, and have associated types (#477)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic

* refacctor map observers

* iterator for map observers

* removed unused ownedptr, added asslice trait to most functions

* make map entry type an associated type

* fix fuzzers

* fix docs

* typo fix

* fix windows, add try_from_slice to shmid

* missing import

* fix fuzzbench

* cleanup

* fmt

* more asslice

* fmt

* added doc link about token-level fuzzing

* cods
2022-01-19 00:02:33 +01:00
Dominik Maier
4f6f76e857
Streamline ShMem API (#472)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic
2022-01-17 18:28:26 +01:00
Andrea Fioraldi
bca1f392a7
Bump to 0.7.1 (#465)
* bump to 0.7.1

* bump libafl_qemu
2022-01-13 11:32:57 +01:00
Dominik Maier
6d9763c51f
Move to clap 3.0 (#447)
* move to clap 3.0

* fix cargo.toml

* update symcc to use clap3
2022-01-04 23:53:12 +01:00
Andrea Fioraldi
37b8cb0d2f Bump to 0.7 2021-12-01 17:22:09 +01:00
Andrea Fioraldi
b4e15fe9f3
Bridge grammartec from Nautilus to libafl (#342)
* nautilus dep

* nautilus generation

* fix mutator

* start new mutator for nautilus

* other mutators

* baby

* ci

* NautilusFeedback

* fix unparse

* ci

* ci

* ci

* ci

* nigghtly clippy

* ci

* fix

* ci

* ci

* update construct automatata

* fix

* ci

* clippy

* clippy

* nightly clippy

* more clippy

* minor clippy

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-11-06 02:21:53 +01:00
Jordan McLeod
3d436b7519
Upgrade to Rust 2021 Edition (#340)
* Enable missing const_xxh3 feature

* Move to Rust 2021 Edition

* Fix clippy complaints

* Remove imports made unecessary in 2021 edition
2021-11-04 09:59:49 +01:00
julihoh
52580c6c14
fix concolic nofloat filter (#333)
* fix concolic nofloat filter

* fmt

* whoops

* fix warnings
2021-10-25 13:57:49 +02:00
Dominik Maier
3f1130a8a4
fixes for frida mode for win and checks in rust 1.56 (#334)
* fixes for frida mode for win

* missing bracket

* fix docs

* fix docs, add windows ci

* disable breaking ci
2021-10-25 11:49:34 +02:00
julihoh
7930d42948
update packages related to concolic (#325)
* update packages related to concolic

* install z3 on CI macOS

I have no clue how this worked before...

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-10-11 21:24:37 +02:00
julihoh
6e2aa47285
add ability to trace location information in concolic tracer (#322)
* add ability to trace location information in concolic tracer

* fix formatting

* introduce location new-type

* fix conolic smoke test

* impl From instead of custom into_inner function

* fmt

* change to use usize instead of NonZeroUsize

in order to no over-constrain the implementation
2021-10-11 13:31:16 +02:00
julihoh
b778f22c0c
don't include all of libafl for symcc_runtime by default, as it's not necessary (#320) 2021-10-11 09:53:15 +02:00
julihoh
6c5f7366ce
concolic optional runtime (#319) 2021-10-10 21:17:40 +02:00
Andrea Fioraldi
2f2634db02
Python basic bindings for sugar and qemu (#302)
* InMemoryBytesCoverageSugar python binding

* InMemoryBytesCoverageSugar python binding

* python mod for qemu in libafl_sugar

* libafl_qemu python

* fix

* clippy fixes

* clippy

* added pyo3-build-config for MacOS builds

* gitignor

* python is not default

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-09-27 09:39:21 +02:00
Dominik Maier
55c4b0c778
added write_file_atomic against ondisk corpus races (#294)
* fix ondisk corpus race condition

* move metadata name to be a dotfile

* note ExitKind for crashes and timeouts in inprocess executor

* potential fix for windows

* added write_file_atomic

* no_std fixes

* no_std testcase fix

* typo fix, windows

* clippy

* more no_std testing
2021-09-13 15:38:28 +02:00
Andrea Fioraldi
2301335bf7 Bump symcc_runtime to 0.1.2 2021-08-23 11:32:03 +02:00
julihoh
bdd1412fc4
Symcc runtime docsrs fix (#270)
* add symcc as submodule

* make build script docs.rs aware

* bump version of symcc_runtime
2021-08-23 09:55:13 +02:00
julihoh
dcf107a30a
symcc_runtime dependency fix for next libafl version & bump to 0.6.0 (#268)
* symcc_runtime dependency fix for next libafl version

* bump to 0.6.0

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-08-18 10:57:55 +02:00
julihoh
7581885d52
prepare concolic support crates for publishing to crates.io (#250)
* prepare concolic support crates for publishing to crates.io

* provide some bare-bones READMEs
2021-08-06 17:49:57 +02:00
julihoh
2282b09ef5
fix warnings hopefully (#245) 2021-08-05 21:57:47 +02:00
julihoh
1dddf7e677
Separate symcc crate (#244)
* create a separate crate for symcc url and commit hash

also contains functions to checkout and build symcc from a build script

* fix dockerfile

* clippy
2021-08-05 21:57:39 +02:00
julihoh
704830a501
Documentation and Refactors for Concolic (#243)
* document symcc_runtime

* rename serialization format expressions to be more concise

* authorship notes

* document dump_constraints

* document smoke test

* tests for serialization format and refactoring

* remove unused bswap message

* remove obselete SymExpr::End

* document and refactor serialization format

* fmt

* more missing docs

* typos

* clippy
2021-08-05 15:01:14 +02:00
julihoh
3d98d31712
Concolic Tracing (#160)
* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* fixed build, clippy

* no_std

* implement WithObservers executor as discussed

* add symqemu as a submodule

* fix symqemu submodule URL to be relative

* update the concolic runtime to match the new interface

* update the trace file header regularly to save constraints in case the program crashes

* add build dependencies for symqemu

* handle full mesage buffer properly

* better policy for updating trace header

* less aggregiously inefficient GC information serialization

* move concolic runtime hitmap count to filter

this is in preparation for the new runtime interface

* very WIP new runtime interface

* use more convenient types in rust runtime

* EmptyRuntime -> NopRuntime

* hide cpp_runtime and formatting

* implement tracing runtime using new runtime interface

* implement filters with new runtime interface

* use a local checkout for symcc_runtime

* make test runtime tracing

* use test_runtime in smoke test

* fix formatting

* make the clippy overlord happy?

* disable symcc build on everything but linux

* make more of symcc_runtime linux only

* fix linking symcc_runtime with C++ stdlib

* will clippy ever be happy?

* formatting

* don't export symcc runtime when compiling tests

* clippy...

* "don't export symcc runtime when compiling tests" for runtime crate as well

* clippy

* move command executor to LibAFL

* move concolic crate into LibAFL

* move concolic{metada,observer} into LibAFL

* move ConcolicFeedback into LibAFL

* move ConolicStage into LibAFL

* fix bug in symcc part of concolic runtime

* stb_image fuzzer with concolic as example fuzzer

* clean up basic_concolic_fuzzer

* clean up and document concolic example fuzzer

* formatting

* clippy

* remove basic_concolic_fuzzer (it is now part of the examples)

* remove the runtime crate in favor of symcc_runtime

* re-architect concolic smoke test and remove git submodules

* remove old submodule directories

* make coverage filter public

* focker docker build

* clippy

* clippy fixes

* fix ubuntu as well

* remove .gitmodules

* move concolic mutational stage into libafl behind feature flag

* script to install dependencies for concolic smoke test

* fix bug

* clippy

* add github action to run smoke test

* fix action

* ensure smoke test is run in correct directory

* remove devcontainer files

* address feedback

* clippy

* more clippy

* address more feedback

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-08-05 13:22:00 +02:00