47 Commits

Author SHA1 Message Date
Dongjia "toka" Zhang
5d7fd8f914
TinyInst Binary-Only fuzzing for Windows (#854)
* step1 for tinyinst

* step2: minimal executor

* updated libafl

* Tinyinst Update (#853)

* Mac OS Autotokens (#723)

* mac_tokens

* more

* win fix

* fmt

* fmt c

* Use nightly fmt (#728)

* Fix compilation for aarch64 qemu (#731)

Typo lead to fail to compile for arm64

* Simd Fix (#729)

* simd fix

* fmt

* Fixing readme & docs (#730)

* fix

* fix

* add

* add

* fmt

* 0.8.1 (#732)

* New Pass Manager Arguments (#724)

* new pm arguments

* enable abgeana's code

* Fix tui with 1 client (#734)

* unbreak tui with 1 client

* clippy

* Add core affinity support for FreeBSD (#736)

* NYX Executor (GSoC '22) (#693)

* Add ccache

* Update codecov.yml

* Add libnyx

* Fix

* Add nyx build script

* Fix build.sh && init executor.rs

* Fix commit

* Fix code

* initialize `exector.rs`

* refine API in `nyx_bridge.rs`

* initialze `run_target`

* add `test_nyxhelper`

* initize `test_executor`

* remove `nyx_beidge.rs`

* make `test_executor` compile

* Improve test

* refine code

* update version

* fix docker

* fix docker

* Fix clippy

* Fix build

* fix build && add `set_timeout`

* Fix and refine CI

* fix CI

* Fix CI

* Add platform restrict

* cargo fmt

* add parallel mode

* add example `nyx_libxml2_parallel`

* fix fuzzer example

* fix CI

* add README

* fix CI

* fix CI

* fix CI

* remove unwrap and NyxResult

* code format fix

* add libnyx's rev

* fix format

* change Duration format && Fix CI

* caego fmt

* fix CI

* fix CI

* Add doc

* test CI

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* add cache for apt and cargo-install

* Update build_and_test.yml

* Update build_and_test.yml

* tmp test CI

* fix CI

* remove debug cmd

* remove test

* code refine

* code refine

* code refine

* code refine

* add Makefile

* fix example doc for nyx

* add `NyxHelper::new_with_initial_timeout`

* fix `NyxHelper::new`

* fix curl parameter

* code refine

* add check for setup script

* use afl-clang-fast in nyx

* fix logic

* fix makefile

* fix CI

* Update build_and_test.yml

* Update build_and_test.yml

* remove debug cmd

Co-authored-by: syheliel <syheliel@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>

* Fix spelling error (#745)

* OSX force_load option (#743)

* Update clang.rs

* fmt

* Add continous JSON Logging monitor (#738)

* Add simple JSON Monitor

* Add documentation

* Log global state

* Fix formatting

* Save state depending on closure outcome, have file opened all the time

* Make OnDiskJSONMonitor cloneable

* Switch to FnMut to allow stateful closures

* Use &mut M: Monitor for the closure

* Fix documentation of Rand::below (#747)

* Netopenbsd build fix (#746)

* core affinity netbsd implementation.

* openbsd build fix

* Fix autotokens doc (#751)

* fix

* remove wrong doc

* Simplification for netbsd-specific code (#750)

the cpuset api is already present in libc...

* Add test case minimising stage (tmin) (#735)

* add test case minimising stage

* general purpose minimiser impl, with fuzzer example

* reorganise, document, and other cleanup

* correct python API return value

* correct some docs

* nit: versioning in fuzzers

* ise -> ize

* Implement a corpus minimiser (cmin) (#739)

* initial try

* correct case where cull attempts to fetch non-existent corpus entries

* various on_remove, on_replace implementations

* ise -> ize (consistency), use TestcaseScore instead of rolling our own

* oops, feature gate

* documentation!

* link c++

* doc-nit: correction in opt explanation

don't write documentation at 0300

* better linking

* Skippable stage, generator wrapper for Grimoire (#748)

* Skippable stage, generator wrapper for Grimoire

* more fancy wrapper

* MapFeedback: Adding support for with_name() (#752)

* Adding support for with_name()

* Adding with_name() function description

* dragonflybsd build fix for core affinity. (#753)

supporting most of linux sched api here.

* CI for FreeBSD (#754)

* CI for FreeBSD

* rustup -y?

* fixed path, switched to clippy

* bsd don't source

* added llvm

* clippy

* more yml

* ?

* testing ci

* llvm?

* llvm??

* more llvm, more tests

* fixed testcase'

* mem limits

* more sudo

* reenable all the CI

* Fixes for new Clippy (#755)

* New Clippy fixes for QEMU (#757)

* Core affinity for FreeBSD pinning task to the wanted cpu (#756)

* Do not zero-init struct in QEMU (#758)

* New Clippy fixes for QEMU

* no need to 0-initialize mem

* clippy

* Add doc for libafl_nyx (#759)

Co-authored-by: syheliel <syheliel@gmail.com>

* Adjust NyxExecutor trait bound to HasTargetBytes from HasBytesVec (#760)

* adjust NyxExecutor trait bound to HasTargetBytes from HasBytesVec

* oops actually use HasTargetBytes instead

* libafl_frida: ASan hook adding Apple's memset_pattern* api. (#761)

* Fix cargo doc on windows (#762)

* add doc cfg

* fix nostd docs

* ignore CommandConfigurator doc test execution on non-unix platform

* add cargo doc step pipeline on windows platform

* Enable memset_patter ASan hooks for Apple on libafl_frida (#763)

* Fix forkserver options (#771)

* Stability improve (#773)

* initial

* add

* fmt & fix

* dbg remove

* clp

* clp

* more

* clippy

* del

* fix

* remove unused

* fix

* doc

* Fix doc (#780)

* Add track_stability option to CalibrationStage  (#781)

* add

* Update gramatron.rs

* Update emu.rs

* try

* clp

* Dump registers on freebsd x86_64 (#779)

* Illumos support (#775)

implementing core affinity too.

* Reduce clang warnings for version output in libafl_cc. (#778)

* Extend gramatron recursive mutator (#783)

* Dump registers on NetBSD amd64 (#786)

* Add support for ARMBE8 (#768)

* Changes to build QEMU out-of-tree so that we don't need to clone the repo for each feature combination we build

* Add be support to libafl_qemu

* More config tweaks

Co-authored-by: Your Name <you@example.com>

* [AFLplusplus/LibAFL] dump registers on OpenBSD amd64 (PR #787)

* dump registers on openbsd

* write_crash implementations

* Windows gdiplus (#789)

* Initial steps

* Harness code cleanup

* don't panic on linux in order not to break the CI

* formatting once again

* restored cfg unix to unbreak linux build

* Remove clang download from windows CI (#791)

* Attempt to remove clang 12 setup

* frida_gdiplus added to CI

* Gdiplus comments (#792)

* Attempt to remove clang 12 setup

* frida_gdiplus added to CI

* Redundancy note

* formatting again :\

* mistake of directory name

* Fix len miscalculation in grimoire string replace (#794)

* Fix len miscalculation in grimoire string replace

* ok Rust i was writing JS these days

Co-authored-by: Andrea Fioraldi <andrea.fioraldi@trellix.com>

* Fix doc typos (#796)

* Fix CI  (#798)

* bump (#799)

* Support for write_crash on netbsd (#788)

* Support for bolts::cpu::read_time_counter on arm64 (#790)

* Add ability to use virtual dispatch to StagesTuple (#801)

* Add ability to use virtual dispatch to stagesTuple

* Fix lint

* Adding CPSR register for arm qemu (#800)

* trying to add in observer

* writing test

* got up to running with instrumentation but i still need to get the map

* fixing fuzzer code

* adding tinyinst fuzzer

* adding ffi to store all the map data into vec.

* adding some new things

* adding somewhat state of how i would like it should work

* fixing some things

* alot of false positives.

* fixing before adding args

* updated to use FileInput!

* adding build script to pull tinyinst

* fixing git issue

* writing instruction to run how to run tinyinst fuzzer

Co-authored-by: Dongjia Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
Co-authored-by: Phan Thanh Duy <phanthanhduypr@gmail.com>
Co-authored-by: Nicholas Lang <97475577+nicklangsysdig@users.noreply.github.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: syheliel <45957390+syheliel@users.noreply.github.com>
Co-authored-by: syheliel <syheliel@gmail.com>
Co-authored-by: Aiden Hall <AidenRHall@users.noreply.github.com>
Co-authored-by: Sönke <eknoes@users.noreply.github.com>
Co-authored-by: Sirui Mu <msrlancern@gmail.com>
Co-authored-by: Addison Crump <me@addisoncrump.info>
Co-authored-by: Patrick Gersch <gersch.patrick@gmail.com>
Co-authored-by: Teddy Heinen <teddy@heinen.dev>
Co-authored-by: Vincent <space_white@yahoo.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: expend20 <36543551+expend20@users.noreply.github.com>
Co-authored-by: Andrea Fioraldi <andrea.fioraldi@trellix.com>
Co-authored-by: Ben Davis <ben@thebendavis.net>
Co-authored-by: radl97 <radl97@users.noreply.github.com>

* fix

* fmt

* Submodule

* Submodule?

* Tinyinst Update V2 (#905)

* updated to lastest libafl

* going to replace tinyinst to more like jackalope with tinyinstrumentation

* fixing clippy

* keep working on cpp ffi. sad

* updating litecov to tinyinst. also start making our own litecov

* revert to map instead of list. not sure why its not working

* making fuzzer listobserver

* working with listobserver!:

* cleaning up

* adding cargo make run

* updating cargo for tinyinst

* updating readme

* readme, clippy

* fmt

* fmt

* fix

* fix

* docker

* fix

* fmt

Co-authored-by: Dominik Maier <dmnk@google.com>
Co-authored-by: biazo <eric.l.biazo@gmail.com>
Co-authored-by: Phan Thanh Duy <phanthanhduypr@gmail.com>
Co-authored-by: Nicholas Lang <97475577+nicklangsysdig@users.noreply.github.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: syheliel <45957390+syheliel@users.noreply.github.com>
Co-authored-by: syheliel <syheliel@gmail.com>
Co-authored-by: Aiden Hall <AidenRHall@users.noreply.github.com>
Co-authored-by: Sönke <eknoes@users.noreply.github.com>
Co-authored-by: Sirui Mu <msrlancern@gmail.com>
Co-authored-by: Addison Crump <me@addisoncrump.info>
Co-authored-by: Patrick Gersch <gersch.patrick@gmail.com>
Co-authored-by: Teddy Heinen <teddy@heinen.dev>
Co-authored-by: Vincent <space_white@yahoo.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: expend20 <36543551+expend20@users.noreply.github.com>
Co-authored-by: Andrea Fioraldi <andrea.fioraldi@trellix.com>
Co-authored-by: Ben Davis <ben@thebendavis.net>
Co-authored-by: radl97 <radl97@users.noreply.github.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2022-12-04 19:04:06 +01:00
Andrea Fioraldi
71dd58396c
libafl_qemu_sys and libafl_qemu_build to have bindgen with QEMU (#915)
* build and sys qemu crates

* working libafl_qemu_build

* libafl_qemu_sys

* switch libafl_qemu to use libafl_qemu_sys

* fix

* use sys

* fmt

* mmu lookup

* fix

* autofix

* clippy

* fix

* allow

* cl

* docker

* docker

* fix

* mem access info in mem hooks

* fmt

* fix

* kill libafl_page_size

* fix

* clippy

* default bindings for docs.rs

* macos

* fix arm build

* fix

* plugins

* fix

* fix fuzzer

* Correct PC on breakpoint

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2022-12-02 17:01:28 +01:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
syheliel
758e49ac70
NYX Executor (GSoC '22) (#693)
* Add ccache

* Update codecov.yml

* Add libnyx

* Fix

* Add nyx build script

* Fix build.sh && init executor.rs

* Fix commit

* Fix code

* initialize `exector.rs`

* refine API in `nyx_bridge.rs`

* initialze `run_target`

* add `test_nyxhelper`

* initize `test_executor`

* remove `nyx_beidge.rs`

* make `test_executor` compile

* Improve test

* refine code

* update version

* fix docker

* fix docker

* Fix clippy

* Fix build

* fix build && add `set_timeout`

* Fix and refine CI

* fix CI

* Fix CI

* Add platform restrict

* cargo fmt

* add parallel mode

* add example `nyx_libxml2_parallel`

* fix fuzzer example

* fix CI

* add README

* fix CI

* fix CI

* fix CI

* remove unwrap and NyxResult

* code format fix

* add libnyx's rev

* fix format

* change Duration format && Fix CI

* caego fmt

* fix CI

* fix CI

* Add doc

* test CI

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* add cache for apt and cargo-install

* Update build_and_test.yml

* Update build_and_test.yml

* tmp test CI

* fix CI

* remove debug cmd

* remove test

* code refine

* code refine

* code refine

* code refine

* add Makefile

* fix example doc for nyx

* add `NyxHelper::new_with_initial_timeout`

* fix `NyxHelper::new`

* fix curl parameter

* code refine

* add check for setup script

* use afl-clang-fast in nyx

* fix logic

* fix makefile

* fix CI

* Update build_and_test.yml

* Update build_and_test.yml

* remove debug cmd

Co-authored-by: syheliel <syheliel@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-08-25 10:55:35 +02:00
Dominik Maier
9dfc6aa404
CI and fixes for arm32 no_std build (#511)
* arm32 no_std fixes and clippy

* moved criterion to benches crate

* benches no longer live here
2022-02-01 00:57:58 +01:00
Andrea Fioraldi
b4e15fe9f3
Bridge grammartec from Nautilus to libafl (#342)
* nautilus dep

* nautilus generation

* fix mutator

* start new mutator for nautilus

* other mutators

* baby

* ci

* NautilusFeedback

* fix unparse

* ci

* ci

* ci

* ci

* nigghtly clippy

* ci

* fix

* ci

* ci

* update construct automatata

* fix

* ci

* clippy

* clippy

* nightly clippy

* more clippy

* minor clippy

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-11-06 02:21:53 +01:00
Dominik Maier
12c470a707
Atheris example to fuzz Python Code (#300)
* initial atheris libfuzzer harness

* cmplog, kinda

* added makefile to generic_inmemory

* Makefile for atheris fuzzer

* moved away from clap yaml

* fixed arg parsing

* fuzzing

* ldpreload lib to replace exit with abort

* fixed docker, docs

* fix docker some more

* better documentation

* less commented out important things

* Make makefile less crashy
2021-11-03 10:13:05 +01:00
Andrea Fioraldi
2055eabede
Port gramatron preprocessing to Rust (#341)
* grammatron random mut

* import String from alloc

* gramatron

* grammar preprocess scripts

* clippy

* fix construct_automata.py

* splice mutator

* fix

* clippy

* recursion mutator

* recursion mut in example

* clippy

* fix

* clippy

* grammars

* fix gramatron

* fmt
2021-10-28 10:37:31 +02:00
Toka
c0d40a43e1
remove libafl_tests (#324)
* remove libafl_tests

* fmt

* fix

* fix

* fix
2021-10-10 20:44:49 +02:00
Andrea Fioraldi
21f88b58b5 Pylibafl 2021-09-27 14:09:15 +02:00
julihoh
1dddf7e677
Separate symcc crate (#244)
* create a separate crate for symcc url and commit hash

also contains functions to checkout and build symcc from a build script

* fix dockerfile

* clippy
2021-08-05 21:57:39 +02:00
julihoh
3d98d31712
Concolic Tracing (#160)
* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* add stub runtime that links with symcc common runtime code

* implement tracing runtime to generate message file

* move ShMemCursor to libafl proper

* qualify enum imports to make clippy happy

* fix warnings

* formatting

* update symcc submodule to point to AFL++ org repo

* fix naming of ShMemCursor and remove std requirement

* ensure runtime is named correctly after compilation

* add devcontainer files for easier development

(will be removed later)

* move rust nightly install into devcontainer.json

this makes it run after the container has been built

* dev container: install recommended packages

* switch to building rust runtime from SymCC cmake

* install corrosion in dev container for cmake-cargo integration

* add smoke test for symcc-runtime integration

* update symcc submodule

* add rustfmt to devcontainer

* properly mark the end of a constraint trace

Using a special "End" message

* small tool to dump constraints from a traced process

* extend smoke test to include parsing & printing of constraints

* update symcc submodule

* first draft of expression filters for concolic

* fix type in runtime method name

* update symcc submodule

* implement extensions to serdeany map:

* remove -> Option<T>
* insert_boxed(Box<T>) (avoids allocation if value is already boxed)

* implement std::io::Seek for ShMemCursor

* implement framing for in-memory traces

this allows to efficiently get the length of trace.
this is important for efficiently copying the trace out of the shared
memory region.

* fix for serdeany map

* fuzzer that associates concolic traces with test
case

* ensure runtime can handle 0-expressions

* move metadata, observer and feedback into separate files

* convert executor to command executor and move to separate file

* refactoring and streamlining

* move panic mode configuration to cmake script

* compile cmake from source, because debians version is too old.........

* use separate stage for tracing

* fix dockerfile

* move runtime into the workspace

using prior work on compilation flags from cmake

* actually make use of selective symbolication filter

* update to support latest symcc changes

* implement hitmap for concolic runtime

* clippy

* implement selective symbolization and coverage map for dump_constraints tool

* use concolic runtime coverage for concolic fuzzer feedback

* actually kill process on timeout

* be extra careful after killing process

* increase command executor busy wait to 5ms

* implement concolic tracing stage

* address naming issue

* implement floating point expression filter for runtime

* rename expression filters to be less verbose

* implement expression pruning

* implement ConcolicMutationalStage

* refactor command executor and remove busy loop

* implement generic command executor

* remove debug prints

* refactor + documentation

* refactor

* fixed build, clippy

* no_std

* implement WithObservers executor as discussed

* add symqemu as a submodule

* fix symqemu submodule URL to be relative

* update the concolic runtime to match the new interface

* update the trace file header regularly to save constraints in case the program crashes

* add build dependencies for symqemu

* handle full mesage buffer properly

* better policy for updating trace header

* less aggregiously inefficient GC information serialization

* move concolic runtime hitmap count to filter

this is in preparation for the new runtime interface

* very WIP new runtime interface

* use more convenient types in rust runtime

* EmptyRuntime -> NopRuntime

* hide cpp_runtime and formatting

* implement tracing runtime using new runtime interface

* implement filters with new runtime interface

* use a local checkout for symcc_runtime

* make test runtime tracing

* use test_runtime in smoke test

* fix formatting

* make the clippy overlord happy?

* disable symcc build on everything but linux

* make more of symcc_runtime linux only

* fix linking symcc_runtime with C++ stdlib

* will clippy ever be happy?

* formatting

* don't export symcc runtime when compiling tests

* clippy...

* "don't export symcc runtime when compiling tests" for runtime crate as well

* clippy

* move command executor to LibAFL

* move concolic crate into LibAFL

* move concolic{metada,observer} into LibAFL

* move ConcolicFeedback into LibAFL

* move ConolicStage into LibAFL

* fix bug in symcc part of concolic runtime

* stb_image fuzzer with concolic as example fuzzer

* clean up basic_concolic_fuzzer

* clean up and document concolic example fuzzer

* formatting

* clippy

* remove basic_concolic_fuzzer (it is now part of the examples)

* remove the runtime crate in favor of symcc_runtime

* re-architect concolic smoke test and remove git submodules

* remove old submodule directories

* make coverage filter public

* focker docker build

* clippy

* clippy fixes

* fix ubuntu as well

* remove .gitmodules

* move concolic mutational stage into libafl behind feature flag

* script to install dependencies for concolic smoke test

* fix bug

* clippy

* add github action to run smoke test

* fix action

* ensure smoke test is run in correct directory

* remove devcontainer files

* address feedback

* clippy

* more clippy

* address more feedback

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-08-05 13:22:00 +02:00
Andrea Fioraldi
dfe39e2af7
libafl_sugar (#215)
* fuzzer mod

* libafl_sugar skeleton

* build libafl_sugar

* libfuzzer_stb_image_sugar

* Delete log

* qemu in libafl_sugar

* docker

* macos merda

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-07-20 11:47:33 +02:00
Andrea Fioraldi
d472a1242a
libafl_qemu (#211)
* empty libafl_qemu crate

* fuzzbench qemu fuzzer skeleton

* emu.run() works without bp

* working emu loop

* resolve elf symbols

* running Qemu fuzzer without coverage

* qemu fuzzer with edge coverage

* merge into inprocess::GLOBAL_STATE

* create QemuExecutor and remove QemuEmulator

* qemu hooks and persist edges mapping storing them in State

* windows fix

* add libafl_qemu to workspace

* windows fix

* some clippy

* clippy

* fix fuzzbench_qemu

* fix fuzzbench_qemu makefile

* fuck you macos
2021-07-08 15:21:14 +02:00
Toka
d4410c072a
ForkserverExecutor (#111)
* add Forkserver, Pipe Outfile struct

* add forkserver executor struct, and shmem init

* close pipes in the destructor of Forkserver

* fill pre_exec to write out the inputs

* fix

* read_st, write_ctl

* more handshakes

* wrap Pipe in Arc, fill post_exec

* add Forkserver, Pipe Outfile struct

* add forkserver executor struct, and shmem init

* close pipes in the destructor of Forkserver

* fill pre_exec to write out the inputs

* fix

* read_st, write_ctl

* more handshakes

* wrap Pipe in Arc, fill post_exec

* fix for the lastest HasExecHooks trait

* use Dominik's pipe, remove Arc and temporarily pass RawFd to setstdin but trying to figure out other solutions

* add libafl_tests, put a very simple vulnerable program

* fix

* added forkserver_simple (mostly copy-pasted from babyfuzzer)

* fix test

* handle crash in post_exec

* add README.md

* check exec time to see why it's so slow

* remove double invokation of is_interesting for the obejctive

* make forkserver_simple AFL-like and improve speed

* some debugging help

* do not evaluate feedback if solution

* speedup the things

* working input placement via stdin in Forkserver

* don't call panic! but return errors, rewrite some comments

* use AFLplusplus/afl-cc instead of AFL

* use .cur_input like AFL

* bring the test for forkserver back

* add better README.md message

* failing the initial handshake should return an error

* delete some commented-out code

* format

* format

* ForkserverExecutor needs std and is unix-only for now

* clippy

* OutFile error handling

* fmt

* clippy

* don't build libafl_tests on windows

* fix

* keep test in forkserver.rs simple

* add forkserver_test feature for libafl_tests

* format

* some doc

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-05-25 13:40:00 +02:00
Andrea Fioraldi
e418198e78 workspace default-members 2021-04-28 12:26:59 +02:00
s1341
5c856cccc8
WIP: Implement ASAN instrumentation using frida (#45)
* frida_asan: Implemented initial asan runtime library

* frida_asan: Switch to hashbrown

* Implemented GOT-based hooking to isolate the hooking of the memory functions. Implemented initial ASAN instrumentation

* WIP: Shadowing all used memory. Currently tracking pages using a BTreeSet. Slow AF!

* Add SigTrap to unix_signals and inprocess

* Working frida-asan, almost no speed degradation.

Currently the shadow check is reversed, so it checks only that the shadow is not 0.
We need to implement sub-8-byte checking.

* Format

* Cleanup and formatting

* Sub-qword and 16-byte checks implemented; Fixed unaligned access to QWORD

* Pass the ucontext_t to signal handlers. Initial regdump on crash

* Fix typo

* Make the context argument a mut ref

* Add missing files; Implement initial reporting

* Refactor out gothook; Move safety checkers to dynasm

* Get rid of const assembly blobs no longer needed

* Move to a handler function instead of using SIGTRAP.

This bloats the transformed code, but doesn't seem to have a major impact on performance.

Also, implemented pretty backtraces and assembly output.

* Formatting

* Get rid of all the pinning crap I wasted my day on, We don't need it

* windows fixes

* ashmem

* ashmem_service: server side ready

* ashmem_service: client side ready. Ready for integration

* ashmem_service: changes to UnixShMem to make it 'threadable'

* ashmem_service: format

* ashmem_service: Undo changes to UnixShMem, make the thread own the AshmemService instead; Fix protocol bug

* ashmem_service: working ashmem service. Fix merge issues

* use the newly released capston e 0.8.0; Fix a nasty bug where the afl_area an pc_pointer were reversed. Changed Vectors to Boxed [u8]

* Implement type detection for reporting; Implement double-free/unallocated free checking

* fmt

* Cleanup code a little

* frida-asan: This is an omnibus commit. Should probably have been a bunch of small commits, but I don't have the time/patience.

 - Implemented DrCov support in order to debug a failing harness. This is actually
   generic and should be moved out of libafl_frida.
 - Implemented LIBAFL_FRIDA_OPTIONS env var to pass options to the frida helper,
   to dynamically enable/disable asan and drcov.
 - Implemented memory reuse - after each test case the used pages are recycled and
   can be reused in the next test case.
 - Implemented and tested vectorized instruction instrumentation.
 - Implemented not instrumenting atomic load/store instructions. The cost of
   trying to emulate their behaviour is too high at the moment.
 - Implemented probing of shadow bit to determine the best match for the current
   system.
 - Implemented shadow memory pre-mapping where it is available. We probe for this
   too.
 - Implemented ability to specify a list of modules to instrument on the command
   line. This allows fine-grained control of which modules are instrumented for
   coverage/asan/drcov.
 - Implemented unpoisoning of the Input target_bytes in a pre_exec hook.
 - Added support for zero-sized allocations. We return 0x10 bytes at the moment.
 - Added all known operator new/delete functions to hooks.
 - Added workaround for frida_gum_allocate_near bug.
 - Cleaned up reporting, added reporting for different error types.

* frida-asan: Implement leak detection

* Fix merge issues

* Rebased on dev to get llmp/shmem changes; Clippy fixes

* Add FridaOptions struct

* Add the Custom ExitKind; Get rid of Clone/PartialEq on ExitKind

* Make it possible to recover from an ASAN error

* Add SIGTRAP to crashing signals

* Add back (conditional) crashing on Asan errors.

* Fix too-large immediates in add instruction

* Implement RcShMemProvider, finally fix the EOP bug

* Clear ASAN_ERRORS before each test

* Fix warnings; Fix review issues

* Cleanup prints

* Add timeout to Frida mode

* Make allocation-/free-site backtraces optional

* CPU Context and backtrace (on android/aarch64 atm) on crash

* Make stalker conditional

* Add metadata to solution, and write metadata files

* Add addresses to backtrace; Add reporting of ASAN stack errors; Fix ASAN reporting bugs

* Remove meaningless backtrace on crash

* Fix the x0, x1 load in report

* use upstream color-backtrace

* use __builtin_thread_pointer instead of custom asm

* Don't unwrap ASAN_ERRORS if it isn't some

* Fix bug where we weren't clearing the drcov basicblocks after each run

* Fix bug where we were dropping an ashmem too soon

* Fix OwnedPtr instead of CPtr

* Fix gettls for all archs

* cfg guards for target arch, disabling Frida-ASAN/-DrCov if not on aarch64

* Cargo fmt

* Only panic in options when asan/drcov are turned on; Merge fixes

* gothook only supported on unix

* Fix gettls on msvc

* Another attempt to fix MSVC gettls

* Fix backtrace use

* nostd fixes; warning fixes

* formatting

* Migrate FridaEdgeCoverageHelper into libafl_frida, and rename to FridaInstrumentationHelper

* Clean up uses

* Move DrCovWriter to libafl_targets

* Refactor DrCovWriter to get a vec of DrCovBasicBlocks; formatting

* Update to newer backtrace which supports android with gimli

* windows fixes

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: andreafioraldi <andreafioraldi@gmail.com>
2021-04-28 10:12:49 +02:00
Andrea Fioraldi
2f7bb8d532 baby fuzzer 2021-04-21 14:51:51 +02:00
Andrea Fioraldi
b4e6115d4f fixes for pcguard and value profile 2021-03-29 15:57:27 +02:00
Andrea Fioraldi
1f7be63031 value profile in libafl_targets 2021-03-26 17:53:47 +01:00
andreafioraldi
7564ce1e87 libfuzzer_stb_image with build.rs and win32 fixes 2021-03-26 10:39:02 +01:00
Andrea Fioraldi
1c8cdc76a8 create libafl_targets and start new structure for libfuzzer_libpng 2021-03-23 20:39:23 +01:00
Dominik Maier
7f72d70977
Windows Fuzzing Example (#41)
* windows fuzzer target and minor changes - breaks android support and maybe linux build

* adapted windows example

* removed warnings from build.rs

* fixed build on unix

* fixed no_std

* build fix, silenced warnings

* no_std warning silenced

* clippy fixes

* fmt

* windows fuzzer target and minor changes - breaks android support and maybe linux build

* adapted windows example

* removed warnings from build.rs

* fixed build on unix

* fixed no_std

* build fix, silenced warnings

* no_std warning silenced

* clippy fixes

* fmt

* clippy

* trying to add clang support

* debugging win build

* debugging win build more

* debuggin..

* debuggin....

* more debugging

* giving up

Co-authored-by: richinseattle <richinseattle@gmail.com>
2021-03-23 13:50:22 +01:00
Andrea Fioraldi
61a89f4aa6 skeleton for libafl_cc 2021-03-22 18:29:46 +01:00
s1341
a02cc39bba
Frida Executor Example (#27)
* inprocess: Allow InProcessExecutor to take a function pointer or a closure

* frida: initial working (but slow + buggy) frida helper

Issues:
- it's slow as ****
- there is an Llmp exception after the 227th corpus entry is found
- Cargo.toml lines currently import from a local ../frida-rust dir, as frida-rust is still under development

* inprocess: let the InProcessExecutor take a closure or a function pointer

* frida: working FridaHelper with InProcessExecutor

* frida: Apply suggestions; Move to RefCell; Cleanup warnings

* frida: link libstdc++_static.a on android

* take an FnMut in InProcessExecutor

* adapt libfuzzer_libpng to FnMut in InProcessExecutor

* create FridaInProcessExecutor and FridaEdgeCoverageHelper

* fix frida build.rs

* frida: move gum to main, get rid of lazy_static; use PageProtection enum

* stalker exclude

* frida: implement inline map-update for x86_64

* inprocess: add harness/harness_mut accessors

* format

* remove get_module_size from FridaEdgeCoverageHelper

* frida: implement aarch64 inline map update

* frida: add missing IndexMode

* add timeouts for executors

* move timeouts to observer

* add with_timeout constructor for Observer

* cast to i64 later in pre_exec

* add cfg(unix) guards

* add TimeoutExecutor

* add TimeoutFeedback and send ExitKind::Timeout from the handler

* pass Duration and move timeout stuff to post_exec

* format

* add timeouts to libpng_libfuzzer

* 10 sec timeout

* timeout executor file

* fix timeout executor no_std

* format

* todos

* Win32ShMem

* win32 exceptions

* fixes

* fix win32 build.rs

* fix win32 build.rs

* fixes fro win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* fixes for win32

* inprocess::windows_exception_handler

* inprocess::windows_exception_handler fixes

* windows_exception_handler in InProcessExecutor

* inprocess::windows_exception_handler fix

* fix windows exceptions mapping

* format

* format

* inprocess: Allow InProcessExecutor to take a function pointer or a closure

* frida: initial working (but slow + buggy) frida helper

Issues:
- it's slow as ****
- there is an Llmp exception after the 227th corpus entry is found
- Cargo.toml lines currently import from a local ../frida-rust dir, as frida-rust is still under development

* inprocess: let the InProcessExecutor take a closure or a function pointer

* frida: Apply suggestions; Move to RefCell; Cleanup warnings

* take an FnMut in InProcessExecutor

* adapt libfuzzer_libpng to FnMut in InProcessExecutor

* reenabled ci for prs

* frida: update to frida-rust 0.3.2

* frida: fix buid errors

* frida: fix build_and_test.yml

* frida: uses crates.io for frida-gum and frida-gum-sys

* fix merge errors

* fix typo

* frida: x86_64 now working

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: toka <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-03-22 12:45:38 +01:00
Takayuki Maeda
39d33ce7ff
Fix clippy warnings (#37)
* fix some clippy warnings

* add a actions job for linting

* remove needless line

* add `run-on` to build_and_test.yml
2021-03-21 22:05:25 +01:00
Andrea Fioraldi
3752821ab4 create libfuzzer_libpng_cmpalloc 2021-03-11 10:09:42 +01:00
Andrea Fioraldi
ff99a442e5
Merge branch 'main' into dev 2021-03-01 18:41:45 +01:00
Marcin Kozlowski
959c8f0dd8
Libmozjpeg example added (#15)
* WIP Harness for libmozjpeg

* Taskset removal (wrong invocation, without -c)

* Clean up
Fixed taskset in test.sh

* Docs

* Formatting

* Formatting

* Formatting

* Formatting

* jpeg example now uses a tokens file

* fixed testcases

* fixing build

* fixed more bugs

* metadatas->metadata

* token files

* added doctest test

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2021-03-01 17:54:47 +01:00
Andrea Fioraldi
f18af45d09 libafl_derive proc macro crate 2021-03-01 14:11:37 +01:00
Dominik Maier
d48524168a Moved fuzzer to examples 2021-02-26 08:35:18 +01:00
Andrea Fioraldi
e99c2daa41 libafl package 2021-02-15 16:54:33 +01:00
Andrea Fioraldi
886aa605a8 fix restating mgr routine 2021-02-11 14:46:28 +01:00
Dominik Maier
9d2635a597 added workspace 2021-02-11 00:38:00 +01:00
Andrea Fioraldi
1ec34dbaa8 move folders 2020-11-22 20:17:39 +01:00
Dominik Maier
cf71410843 added autotranslated llmp 2020-11-21 20:57:42 +01:00
Dominik Maier
a4690046b3 updated xxxhash 2020-11-20 22:11:55 +01:00
Andrea Fioraldi
c617f3a397 libfuzzer clone project 2020-11-20 13:28:34 +01:00
Dominik Maier
ec36301414 started moving to no_std 2020-11-15 02:04:13 +01:00
Andrea Fioraldi
604e278664 generic map observer 2020-10-29 13:18:09 +01:00
Dominik Maier
d4de6f86da added sighandlers 2020-10-28 18:43:59 +01:00
Dominik Maier
c0e803baae added initialstrages trait 2020-10-27 15:08:15 +01:00
Dominik Maier
ca1be853e1 code format, infos in cargo toml 2020-10-27 11:45:10 +01:00
Dominik Maier
6e949fa469 moved to xxh3_rust 2020-10-27 10:31:29 +01:00
Dominik Maier
abc3eb8664 started corpus 2020-10-26 03:07:48 +01:00
Dominik Maier
d91717a34c added utils, folder structure 2020-10-23 20:05:08 +02:00
Dominik Maier
bca91aeafb initial commit 2020-10-23 01:49:09 +02:00