1688 Commits

Author SHA1 Message Date
52d927dc61 incomplete merge 2022-12-08 09:15:27 +01:00
Dongjia "toka" Zhang
8444cf7cc8
Fix libafl_qemu i386 build (#924)
* fix

* fmt

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-12-04 15:07:30 +01:00
Andrea Fioraldi
71dd58396c
libafl_qemu_sys and libafl_qemu_build to have bindgen with QEMU (#915)
* build and sys qemu crates

* working libafl_qemu_build

* libafl_qemu_sys

* switch libafl_qemu to use libafl_qemu_sys

* fix

* use sys

* fmt

* mmu lookup

* fix

* autofix

* clippy

* fix

* allow

* cl

* docker

* docker

* fix

* mem access info in mem hooks

* fmt

* fix

* kill libafl_page_size

* fix

* clippy

* default bindings for docs.rs

* macos

* fix arm build

* fix

* plugins

* fix

* fix fuzzer

* Correct PC on breakpoint

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2022-12-02 17:01:28 +01:00
Addison Crump
5252097819
Fix scores in minimizer when using on_replace (#920) 2022-11-30 14:32:26 +01:00
Andrea Fioraldi
c2776e117a
emu::current_cpu() is now the CPU that hitted the breakpoint in fullsystem (#910)
* emu::current_cpu() is now kept after vm stop and it is the CPU that hitted the breakpoint

* clippy

* uninit

* clippy

* clippy

* clippy

* clippy

* nightly override in CI

* nightly override in CI

* components

* components

* targets

* targets

* clippy

* clippy

* clippy

* clippy

* clippy (again)

* MaybeUninit

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-25 11:57:08 +01:00
Langston Barrett
bc85129cd9
libafl_frida: Point to the LibAFL book from API docs (#886)
Help users find the docs!
2022-11-25 01:48:21 +01:00
omergreen
889161e55e
Fix mac m1 incompatibility for cmplog in frida mode (#914)
For some reason, Apple's aarch64 processor throws a SIGILL when encountering LDP x5, x5 (or any other repeating register). STP works, but I changed both for symmetry.
2022-11-25 01:31:22 +01:00
Andrea Fioraldi
3f627aaf0b
Save and restore CPU state in libafl_qemu (#907)
* libafl_qemu: fix systemmode with slirp dependency

libslirp will be dropped from future QEMU releases (see https://wiki.qemu.org/ChangeLog/7.0).
This change adds the "slirp" feature,
which links with the host-systems libslirp.

* libafl_qemu: enable systemmode snapshots, vm_start

Re-enable snapshot functions.
Start the VM before qemu_main_loop.

* libafl_qemu: allow synchronous snapshotting

Add a flag to take snapshots synchronosly.
This should be used to take or load snapshots while the emulator is not
running.

* libafl_qemu: fallback cpu for read-/write_mem

In systemmode, current_cpu may not be set.
In such cases use the first cpus memory access methods.

* fuzzers: add example for libafl_qemu in systemmode

* libafl_qemu: update libafl-qemu-bridge revision

* libafl_qemu: add memory access by physcial address

* fix liabfl_qemu example

Use GuestAddr and physical memory access

* ci: install libslirp-dev for libafl_qemu

* fuzzers/qemu_systemmode: clean up example

* libafl_qemu: remove obsolete functions

emu::libafl_cpu_thread_fn
emu::libafl_start_vcpu
emu::start

* fuzzers/qemu_systemmode: simplify example

* improve build_linux.rs

* Update qemu_systemmode fuzzer

* upd

* clippy

* Save and restore CPU state in libafl_qemu

* clippy

* Clone

* upd

* upd

Co-authored-by: Alwin Berger <alwin.berger@tu-dortmund.de>
2022-11-22 16:29:43 +01:00
Andrea Fioraldi
7b0039606b
Forksrv adaptive map size and AFL++ CmpLog support (#896)
* AFL++ cmplog map

* map size opt in forkserver

* MapObserver::downsize_map and adaptive map size in forkserver

* fix fokserver_simple cmd opts

* clippy

* fuzzbench forkserver with cmplog

* delete makefile in fuzzbench forkserver

* fuzzbench_forkserver is persistent

* ForkserverExecutorBuilder::build_dynamic_map

* fix

* clippy

* fix

* fix macos

* fix compilation

* fix bugs

* fixes

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-22 10:33:15 +01:00
Alwin Berger
b33839708e
Fix QEMU systemmode fuzzing (#883)
* libafl_qemu: fix systemmode with slirp dependency

libslirp will be dropped from future QEMU releases (see https://wiki.qemu.org/ChangeLog/7.0).
This change adds the "slirp" feature,
which links with the host-systems libslirp.

* libafl_qemu: enable systemmode snapshots, vm_start

Re-enable snapshot functions.
Start the VM before qemu_main_loop.

* libafl_qemu: allow synchronous snapshotting

Add a flag to take snapshots synchronosly.
This should be used to take or load snapshots while the emulator is not
running.

* libafl_qemu: fallback cpu for read-/write_mem

In systemmode, current_cpu may not be set.
In such cases use the first cpus memory access methods.

* fuzzers: add example for libafl_qemu in systemmode

* libafl_qemu: update libafl-qemu-bridge revision

* libafl_qemu: add memory access by physcial address

* fix liabfl_qemu example

Use GuestAddr and physical memory access

* ci: install libslirp-dev for libafl_qemu

* fuzzers/qemu_systemmode: clean up example

* libafl_qemu: remove obsolete functions

emu::libafl_cpu_thread_fn
emu::libafl_start_vcpu
emu::start

* fuzzers/qemu_systemmode: simplify example

* improve build_linux.rs

* Update qemu_systemmode fuzzer

* upd

* clippy

Co-authored-by: Alwin Berger <alwin.berger@tu-dortmund.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-11-21 17:57:06 +01:00
Addison Crump
0515eebbd2
Differential observers (#868)
* reduce diffexecutor constraints for new (so it may be used in a manager-less environment)

* add differential observers

* finish differential observeration

* requirement for observers (weak), default impl for time observer

* make the map swapper, revisit how differentialobserver is implemented

* semi-specialise multimap, add example

* improve example slightly

* fix clippy lints

* fix last clippy issue

* better docs + example flow

* improve example: correct map sizing + multimap vs split slice

* correct some comments

* fix tests + slight bit more docs

* fix bindings

* fixups for the CI

* typo fix

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-20 23:56:23 +01:00
Patrick Gersch
556789dffa
Adding DrCov for qemu (#878)
* Adding DrCov for qemu

* Fixing cargo fmt

* Trying to fix maturin build

* Fixing clippy

* libafl_qemu --no-default-features fix

* Adding make module mapping a user input as suggested from @WorksButNotTested

* Switching from blocks_raw() -> blocks() and full_tracing as an option

* Avoiding get before get_mut

* HashSet to Vec

* Avoiding lazy_static

* Adding DrCov for example fuzzer qemu_arm_launcher

* Removing mut for globals in DrCov

* Using emu.mappings() for drcov module mappings

* Fixing clippy

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2022-11-20 14:28:30 +01:00
Jorgecmartins
bd62cebd7e
Fix typo in observer.md (#904) 2022-11-20 14:28:03 +01:00
David CARLIER
32dc796234
Extend autotokens pass support to other unixes (#900) 2022-11-20 09:24:30 +01:00
Mrmaxmeier
ff2971068f
mopt: seed from state rand instead of current_nanos (#902) 2022-11-20 09:21:16 +01:00
julihoh
948c94d695
Update and fix concolic support (#901)
* fix incorrect assert condition and document it

* update symcc

* adapt to changes in symcc API

* more fixes

* fix formatting

* more fixes

* speed up smoke test by building multiple crates in one command

* update symcc commit to latest main
2022-11-19 23:05:15 +01:00
David CARLIER
f7f6392a4b
forkserver support attempt on freebsd (#898)
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2022-11-18 07:14:15 +09:00
David CARLIER
d77769540a
fixing freebsd unused import warning in core affinity. (#897)
fixing clang wrapper test passing llvm pass api support when there are actual passes.

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2022-11-17 10:28:22 +01:00
Dongjia "toka" Zhang
ec38858b2d
Fix Makefile.toml (#893)
* don't use submodules

* fix

* add

* fix

* a

* fix

* doesn't work 😩

* fix

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml
2022-11-17 04:44:26 +09:00
Dominik Maier
a22c76e02e
Improve Apple support for libafl_cc dll_extensions (#892) 2022-11-15 18:28:52 +01:00
Dominik Maier
2011ed299b
Pthread introspection hook (extends #263) (#891)
* Add pthread_introspection_hook support on macos

See-also: #68

* Remove lazy_static

* all of apple are created equal

Co-authored-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
2022-11-15 18:27:48 +01:00
Dominik Maier
e5aaf85d3c
Tuneable Stage, Scheduler, ScheduledMutator (#874)
* Tuneable Stage, Scheduler, and Mutators

* rename

* fix build

* get rid of prelude

* fmt

* Reworked tunable, fixed stuff, add testcase

* clippy

* further fixes

* fix typo, fmt
2022-11-12 03:02:54 +01:00
Andrea Fioraldi
fe459f6fa5
DumpOnDiskStage in fuzzbench_text to dump the grimoire inputs as bytes for the fuzzbench measurers (#869)
* FuzzbenchDumpStage in fuzzbench_text

* fix

* DumpOnDiskStage

* clippy

* removed duplicated code from example fuzzer

* shorthand to move OwnedSlice into vec

* clippy

* fiz

* fix missing semicolon

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-11 17:38:48 +01:00
Dominik Maier
e340d35674
Add standalone toolchain link to frida_libpng (#890) 2022-11-11 16:06:38 +01:00
Lukas Seidel
17a0d9e8f0
Forkserver: Add file input support (#880)
* make use of clap derive in forkserver_simple

* (re)introduce use_shmem_testcase flag to ForkserverExecutor

* set use_shmem_testcase flag automatically based on forkserver handshake

* remove illegal_state and just .unwrap instead as the None case is unreachable

* fix: removed pub method

* cargo fmt

* remove illegal_state #2 and just .unwrap instead as the None case is unreachable

* change shmem unwrap to unwrap_unchecked

* fix double mut

* removed @@ warning
2022-11-10 15:25:52 +01:00
Dominik Maier
977415cad2
Reworked Book, add missing files (#888)
* Added missing links to docs

* Reworked docs

* Remove empty file

* remove Launcher info (that moved to spawn_instances)

* ignore more
2022-11-10 13:08:35 +01:00
Langston Barrett
893f284482
Use bytes, not strings, for stdio observers (#885)
Previously, the `CommandExecutor` attempted to decode its child
process's stdout and stderr as UTF-8 `String`s. This could fail
if the output was not UTF-8. However, the `Std{Out,Err}Observer`s
should probably be able to be used in such a situation - Consider
fuzzing `echo` with a random `BytesInput`.

The fix is to not decode the output, but rather directly store and
provide the bytes of stdout/stderr in the observers.
2022-11-10 12:40:59 +01:00
Dominik Maier
1486c204eb
Remove unused stage stub (#882)
* GetDeps Stage

* removed getdeps stage
2022-11-10 09:39:29 +01:00
Dominik Maier
6b6570ae5f
Use Associated Types for Has* traits and AsSlice (#881)
* More Associated Types

* AsSlice associated-ified

* added script to find unused files

* fixes for python

* build all the things

* windows fixes
2022-11-10 09:31:04 +01:00
Peter Whiting
18f288e2d3
Monitor to export fuzzer metrics to Prometheus server (#875)
* add custom monitor prometheus as a baseline to build functionality

* working server, set up function to update metrics in the registry

* for a test

* metrics for corpus count, objective count, executions, execution rate are intermittently updated and exposed on /metrics

* add runtime metric, clean up some comments

* IP:PORT as argument instead of hardcoded

* add client # as label attached to fuzzer metrics for filtering by client. add clients_count as a tracked metric

* added support for custom metrics added to client_stats via feedbacks, such as edges count. cleaned up code

* cargo fmt

* clean up prometheus.rs

* ran autofix and fmt scripts, and put optional dependencies behind prometheus_monitor feature

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2022-11-07 19:01:59 +01:00
Mrmaxmeier
8e6eaf7002
check_for_blobs.sh: respect gitignore (#876) 2022-11-05 23:30:26 +01:00
David CARLIER
b9bd0dd6b7
sort of fix core affinity on mac arm64 (#873)
* sort of fix core affinity on mac arm64
we can t pin to a coreid however we can at least choose the performance
cores for our thread.

* using other cores as well

* Fix CI yml (#871)

* Fix CI again (#872)

* Fix CI yml

* Fix CI

* Add dump_register/write_crash for freebsd arm64 (#870)

Co-authored-by: Dominik Maier <domenukk@gmail.com>

* Remove QEMU-Nyx & packer submodules

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2022-11-05 21:52:43 +09:00
Dongjia "toka" Zhang
dddfaf3f55
Remove QEMU-Nyx & packer submodules 2022-11-05 12:43:36 +09:00
David CARLIER
79fc952f53
Add dump_register/write_crash for freebsd arm64 (#870)
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2022-10-29 09:40:25 +02:00
Dominik Maier
ff945d9657
Fix CI again (#872)
* Fix CI yml

* Fix CI
2022-10-29 00:05:28 +02:00
Dominik Maier
17d0795f0d
Fix CI yml (#871) 2022-10-28 23:50:38 +02:00
Andrea Fioraldi
40269a578b
Delete blob and add CI check (#867)
* Delete blob and add CI check

* fix

* shellcheck
2022-10-27 16:20:29 +02:00
Andrea Fioraldi
ebdab32b36
Fix clap in fuzzbench fuzzers (#866) 2022-10-26 14:06:14 +02:00
Andrea Fioraldi
5da5997b20
Remove fuzzbench_weighted and update fuzzbench (#865) 2022-10-26 11:24:34 +02:00
David CARLIER
3054a69cf6
Dump_registers update on netbsd x86_64 arch. (#863) 2022-10-26 09:51:45 +02:00
Andrea Fioraldi
31077765de
Fix CI (#862)
* Autofix with new clippy

* Clippy
2022-10-26 09:41:08 +02:00
Patrick Gersch
cf9c4188c0
Disabling qemu dependecies for qemu fullsystem (#737)
* Disabling qemu dependecies by default

* Adding full emulation_mode support

* Removing usermode from libafl_qemu default features

* Fixing refactoring

* Fixing typo in systemmode

* Fixing clippy:needless-borrow

* Mark libafl_load/save_qemu_snapshot as unused + cpu_reset

* Fixing clippy::needless-borrow

* Fixing needless-borrow yet again

* reset_cpu -> cpu_reset

* Fixing github workflow yet again

* Fixing clippy::uninlined-format-args

* Adding current libafl_qemu_bridge

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-10-25 14:16:11 +02:00
Andrea Fioraldi
5571a03641
Implement thread-safe AsanGiovese in Rust with snapshots support (#851)
* Purge C impl of asan-giovese

* Compiling

* reset asan

* Restore asan state in qemu

* clippy

* upd

* Asan snapshots

* fuzzbench_qemu

* fix snap mmap limit

* fix

* compiles again

* clippy

* update meminterval

* autofix

* fix 32 bit targets

* try to clean intermediate builds

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-10-25 09:48:59 +02:00
Sönke
332c2bc3f8
Fix launcher to work with returning run_client functions (#860) 2022-10-24 21:40:24 +02:00
Andrea Fioraldi
1eb738695f
Fix stdio observer refactor (#859) 2022-10-24 13:50:11 +02:00
Dominik Maier
de99ee1340
Doc fix (#857) 2022-10-24 03:32:16 +02:00
Dominik Maier
663a33168e
Associated types for Corpus, State (#767)
* Associated types for Corpus, State

* cleanup

* fix no_std

* drop unused clauses

* Corpus

* cleanup

* adding things

* fixed fuzzer

* remove phantom data

* python

* progress?

* more more

* oof

* wow it builds?

* python fixes, tests

* fix python fun

* black fmt for python

* clippy, added Nop things

* fixes

* fix merge

* make it compile (#836)

* doc-test fixes, prelude-b-gone for cargo-hack compat

* fixes for windows, concolic

* really fix windows, maybe

* imagine using windows

* ...

* elide I generic when used with S: State

* Elide many, many generics, but at what cost?

* progress on push

* Constraint HasCorpus, HasSolutions at trait definition

* remove unused feature

* remove unstable usage since we constrained HasCorpus at definition

* compiled, but still no type inference for MaxMapFeedback

* cleanup inprocess

* resolve some std conflicts

* simplify map

* undo unnecessary cfg specification

* fix breaking test case for CI on no-std

* fix concolic build failures

* fix macos build

* fixes for windows build

* timeout fixes for windows build

* fix pybindings issues

* fixup qemu

* fix outstanding local build issues

* maybe fix windows inprocess

* doc fixes

* unbridled fury

* de-associate State from Feedback, replace with generic as AT inference is not sufficient to derive specialisation for MapFeedback

* merge update

* refactor + speed up fuzzer builds by sharing build work

* cleanup lingering compiler errors

* lol missed one

* revert QEMU-Nyx change, not sure how I did that

* move HasInput to inputs

* HasInput => KnowsInput

* update bounds to enforce via associated types

* disentangle observers with fuzzer

* revert --target; update some fuzzers to match new API

* resolve outstanding fuzzer build blockers (that I can run on my system)

* fixes for non-linux unixes

* fix for windows

* Knows => Uses, final fixes for windows

* <guttural screaming>

* fixes for concolic

* loosen bound for frida executor so windows builds correctly

* cleanup generics for eventmanager/eventprocessor to drop observers requirement

* improve inference over fuzz_one and friends

* update migration notes

* fixes for python bindings

* fixes for generic counts in event managers

* finish migration notes

* post-merge fix

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2022-10-24 03:22:26 +02:00
Dominik Maier
9695ce0029
Refactor Output Observers (#856)
* Refactor Output Observers

* Delete .gitmodules

* modules

* Drop need for OutputObserving list
2022-10-24 02:50:00 +02:00
Dongjia "toka" Zhang
5b75b6b8ac
Set persistent mode env variables. (#852)
* persistnt mode envs

* clp

* clpgit add -u!
2022-10-23 20:35:35 +02:00
Alessandro Mantovani
0307dadcd1
Allow two different observers for DiffExecutor (#843)
* DifferentialExecutor for CommandExecutor along with StdIO observer

* format

* fix CI issues

* fix format and unit test

* fix documentation

* allow three structs and doc only for linux

* resolve documentation test failure

* minor

* running fmt_all.sh

* into_executor() takes 4 params, not just 1

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2022-10-23 01:59:40 +02:00