37 Commits

Author SHA1 Message Date
Dongjia "toka" Zhang
6f21cb3848
Bump to 0.10.1 (#1280)
* bmp

* remove DEBUG env var
2023-05-22 12:42:36 +02:00
Langston Barrett
aa3f126100
LibAFL_qemu: Return errors from Emulator::new instead of asserting (#1197)
* qemu: Return errors from Emulator::new instead of asserting

Libraries should not `assert!` except in cases of unrecoverable (library)
programmer error. These errors are all potentially recoverable, and aren't
internal errors in `libafl_qemu` itself.

* Respond to review comments
2023-04-09 21:27:27 +02:00
Andrea Fioraldi
0f633962ff
Bump to 0.10.0 (#1156)
* Bump to 0.10.0

* fix

* Fix CI

* Fix copyright

* fmt

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-04-05 17:49:53 +02:00
Dongjia "toka" Zhang
ccd1211cd2
Remove qemu_arm_launcher test from CI (#1179)
* remove qemu arm

* trigger all

* debug

* revert

* api chg

* fix

* try

* debug

* remove qemu

* no_std

* Update build_and_test.yml

* llvm16

* revert z3

* macos

* fix

* remove test

* don't grep on mac

* fix

* ok

* f

* f

* f

* aaa
2023-04-04 14:49:58 +02:00
Dominik Maier
bbe4e85768
Removed new_ from constructors that don't need it (API consistency) (#1159)
* Removed new_ from constructors that don't need it (API consistency)

* un-change python bindings
2023-03-17 17:02:21 +01:00
Vincent
c8254dbd0e
Check CI result on cargo make test for available fuzzers (#1107)
* fix libfuzzer_libpng_cmin

* fix libfuzzer_libpng_ctx

* revert libfuzzer_libpng_cmin and check ci by grepping broker stdout result instead

* revert libfuzzer_libpng_ctx and check ci by grepping broker stdout result instead

* add check ci for fuzzers

* add check ci for fuzzers

* add check ci

* add fuzzbench test

* add validation for qemu fuzzer

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-03-06 23:21:24 +09:00
R. Elliott Childre
5d76707ede
Bump deps and fix Clippy warns in example fuzzers (#1043)
* Mostly addressing changing the `uninlined_format_args` lint which was
  changed to warn-by-default in rust clippy 1.67

* Bump dependencies:
  bindgen:  0.61 -> 0.63
  cc:       1.0 -> 1.0.42 (Exclue versions w/incompat rayon dependency)
  clap:     3.x -> 4.0
  rangemap: 0.1 -> 1
  xz -> xz2:  move to updated version

* Add fallthrough default return to `LLVMFuzzerTestOneInput` in
  **/fuzz.c to prevent Clang's -Wreturn-type

* libafl_atheris: Improve POSIX compatibility and reduce warnings
  * Check for .dylib and .so libraries
  * `source` -> `.` for POSIX shells
  * install wheel into the venv to support newer Python packaging
    standards
  * `LDPRELOAD` -> `LD_PRELOAD`
2023-02-05 21:53:45 +01:00
Andrea Fioraldi
eaf5ff9de0
Restart loading initial inputs even after a crash/timeout (#1040)
* Track initial inputs loading

* libfuzzer libpng

* fuzzbench

* fix no_std

* fix no_std

* clippy

* fuzzers
2023-02-03 11:56:47 +01:00
Andrea Fioraldi
fdf579bcd5
Bump to 0.9.0 (#946)
* bump to 0.9.0

* fix libafl_tinyinst

* fix

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-01-31 10:42:01 +01:00
Dominik Maier
97e88af0c5
OnDiskCorpus: Write metadata by default, metadata gzip compression (#995)
* Write metadata by default

* fix fuzzers

* Cleanup, gzip feature

* Fix casing for ondisk corpus

* fix fmt, clippy

* clippy

* clippy for gdiplus fuzzer

* fmt
2023-01-13 01:07:36 +01:00
Dominik Maier
75f12bd0eb
Remodelling Observers/Examples that rely on UB, API cleanups (#950)
* Tackling UB

* PtrMut -> MutPtr, moved mapobservers to non-UB

* QEMU fixes

* test fixes

* qemu

* Change all interfaces, fix all fuzzers

* fixes

* fix more fixes

* fmt

* fix qemu sugar

* fix some qemus

* atheris

* fmt

* more fmt

* most fmt

* more fix

* nyx fyx

* fix qemu

* clippy, fixes

* more fixes

* no unfix, only fix

* fix

* fix

* more clippy

* fixes

* ListObserver

* fmt, clippy

* fix qemu on arm

* update zlib target

* fix?

* fix

* added migration guide

* ignore doc

* fix symcc

* fix new win fuzzer

* Fixes, rename PTR_SIZE to PTR_NUM

* Try fix linking on win

* Trying to fix win linking

* more cov

* trying to fix win some more

* trying to fix mac

* trying to fix mac

* Fix tests

* Fix tests

* trying to fix win

* more mac

* giving up for windows

* fmt

* python3

* mac?

* undo windows tests
2022-12-24 14:20:44 +01:00
Patrick Gersch
cf9c4188c0
Disabling qemu dependecies for qemu fullsystem (#737)
* Disabling qemu dependecies by default

* Adding full emulation_mode support

* Removing usermode from libafl_qemu default features

* Fixing refactoring

* Fixing typo in systemmode

* Fixing clippy:needless-borrow

* Mark libafl_load/save_qemu_snapshot as unused + cpu_reset

* Fixing clippy::needless-borrow

* Fixing needless-borrow yet again

* reset_cpu -> cpu_reset

* Fixing github workflow yet again

* Fixing clippy::uninlined-format-args

* Adding current libafl_qemu_bridge

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-10-25 14:16:11 +02:00
Andrea Fioraldi
5571a03641
Implement thread-safe AsanGiovese in Rust with snapshots support (#851)
* Purge C impl of asan-giovese

* Compiling

* reset asan

* Restore asan state in qemu

* clippy

* upd

* Asan snapshots

* fuzzbench_qemu

* fix snap mmap limit

* fix

* compiles again

* clippy

* update meminterval

* autofix

* fix 32 bit targets

* try to clean intermediate builds

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-10-25 09:48:59 +02:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
Dongjia Zhang
eac7307c5a
0.8.1 (#732) 2022-08-18 10:23:57 +02:00
Dongjia Zhang
ffe8dbf6af
Bump to 0.8.0 (#696)
* upd

* more
2022-07-11 21:59:11 +02:00
Andrea Fioraldi
7147170240
New hooks for libafl_qemu (#673)
* new block and edge hooks

* Wrking new hooks

* no Pin, just box

* working call tracing

* invalidate_block flag

* working call stack tracking helper

* callstack push

* fixes

* py

* fixes

* clippy

* clippy

* gdb api

* kill introspection

* fix

* upd qemu

* upd qemu
2022-06-16 11:09:07 +02:00
Dominik Maier
3a5118fc02
Moved core_affinity to bolts (#655)
* moved core_affinity to bolts crate

* clippy

* fixes

* ubuntu

* ubuntu++

* moved core_affinity to os

* fixed more imports

* fixed imports

* fixed test

* moved core_affinity out of os

* added affinity

* moved to windows crate

* fmt

* some tiny fixes

* more win

* refactoring

* win?

* win?

* clp

* upd

* more

* copy & paste & fix

* clp

* try

* fix

* more

* fix

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2022-05-30 10:02:46 +02:00
Dominik Maier
bfe69aea09
Format C/Cpp code in ./scripts/fmt_all.sh (#653)
* format all (clang format 13)

* added clang-format check

* re-add missing newline

* cargo doc was missing

* more brackets

* fixed fmt workflow

* clang format

* shellcheck

* install clang-format-13

* update ubuntu for maximum clang-formattability

* yml whitespaces

* fmt

* shellcheck only for .sh

* oops path

* ignored shellcheck warning
2022-05-29 03:23:02 +02:00
Andrea Fioraldi
a544bc042d
Move build_id to bolts (#649)
* Drop the build_id depedency and move to bolts

* tabs->spaces

* clippy build_id fixes

* frida clippy

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-05-27 01:05:03 +02:00
Andrea Fioraldi
da537aae83
FeedbackState as metadata (#627)
* SerdeAny MapFeedbackState

* Fix macro syntax

* alloc

* fix

* Metadata calibrate and map feedback

* metadata feedback states

* compile

* fmt

* Register common generic types

* tests

* sugar

* no_std

* fix book

* alloc

* fix fuzzers

* fix

* fmt

* disable python bindings for libafl

* clippy

* fmt

* fixes

* fmt

* fix

* fix

* fix

* fix

* fix

* release autofix

* fix

* fix

* fix

* fmt

* fix

* fix

* name

* fix

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-05-24 16:05:22 +02:00
Dominik Maier
5df130188a
Fixing CI from #559 (#580)
* updated ref

* update symcc

* updated symcc

* CI for symcc

* updated symcc

* enabling git

* add runtime deps to makefile

* only linux

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2022-03-29 14:57:38 +02:00
Andrea Fioraldi
a56f4af7da
CorpusScheduler -> Scheduler and move them to the schedulers folder (#560)
* CorpusScheduler -> Scheduler

* fix book

* update fuzzers

* fix tests

* fix sugar

* fix

* fix tutorial

* fix tutorial

* fmt

* fix

* fmt

* fmt
2022-03-03 14:27:37 +01:00
Dongjia Zhang
ba4cca0e15
Delete redundant makefiles (#546)
We switched to cargo make
2022-02-20 18:50:29 +01:00
Dongjia Zhang
fc89f2944b
Makefile.toml fix (#545) 2022-02-20 04:21:43 +01:00
Dongjia Zhang
936e2221d1
Cargo-make (#537)
* timeout utility

* example build.toml

* upd

* ci

* Update build_and_test.yml

* Update build_and_test.yml

* rename, qemu_launcher

* libpngs

* fix

* upd

* del

* do_nothing -> unsupported

* rename

* use command

* non qemu fuzzbench

* script.sh

* mroe

* qemu

* fix

* generic

* fix

* fix

* allow 124

* quotes

* fix

* fix

* fix

* stderr to devnull

* chg
2022-02-20 03:32:43 +01:00
Andrea Fioraldi
a03d733cf9
libafl_qemu decouple hooks from the executor and QemuForkExecutor (#528)
* QemuHooks

* option state hooks

* QemuForkExecutor

* enforce no side effects in QemuForkExecutor

* child hooks fixes

* fixes

* qemu_launcher

* examples and fixes

* fix sugar

* clippy

* fmt

* no timeout for fuzzbench_fork_qemu

* Update libafl_qemu/src/hooks.rs

Co-authored-by: Alwin Berger <50980804+alwinber@users.noreply.github.com>

* clippy

Co-authored-by: Alwin Berger <50980804+alwinber@users.noreply.github.com>
2022-02-15 22:11:24 +01:00
Andrea Fioraldi
dd002a081b
Implement coverage accounting (BB metric atm) (#507)
* bb accounting llvm pass

* bb metric

* accoutning corpus scheduler

* fix warnings

* alloc

* clippy

* fix dockerfile

* clippy

* coverage accounting example

* finish CoverageAccountingCorpusScheduler

* fmt

* --libs in llvm-config

* merge
2022-02-01 14:08:38 +01:00
Youssef
e307dfb16f
Implement backtrace observers for crash dedupe (#379)
* create stacktrace observer

* create stacktrace feedback

* post-merge fixes

* address comments

* update Cargo.toml

* fix CI issue + dynamic naming

* duplicate baby_fizzer

* update stacktrace baby_fuzzer

* force unwinding tables

* ignore test dumps

* fix stacktrace baby_fuzzer logic

* upgrade Backtrace version

* trigger observers.post_exec in crash_handler

* implement NewHashFeedbackState and update logic

* digest symbols pointers

* cleanup

* minimal output

* fix backdated EventFirer generic param

* add baby_fuzzer example with a fork executor

* duplicate baby_fuzzer_stacktrace with forkexecutor

* backtrace collection implemented

* add c app fuzzer example with a fork executor

* group backtrace baby fuzzers

* added c code baby fuzzer with inprocess executor

* remove need for static COLLECT_BACKTRACE

* moved code to stacktrace.rs + fixed bug

* add comment

* add command executor fuzzer example

* post merge cleanup

* add missing doc

* address comment

* fix nit

* clean duplicate variable in timeout handler

* fix command executor bt collection

* clean code and use StdShMem

* cleanup

* add ObserverWithHashField + rename StacktraceObserver

* rename + refactor some code

* add CommandBacktraceObserver

* update command executor

* update baby fuzzers

* simplify BacktraceSharedMemoryWrapper

* use better names + static methods

* use std feature macro on BacktraceObserver + fix bug

* use Box in HashValueWrapper to minimize variants size diff

* use copy_from_slice

* std conditional backtrace collection

* fix std import

* fix comment

* add exit_kind to observer.post_exec

* added hash trait to Input

* collect backtrace in post_exec

* add crash handlers to InProcessForkExecutor

* fix panic message

* duplicate forkserver fuzzer example

minimal example

update

* proto bt collection working

* rename CommandBacktraceExecutor to ASANBacktraceExecutor

* refactor ASANBacktraceObserver

* support for forkserver working

* update fuzzer example

* less verbosity

* Post merge fixes

* implement hash for GeneralizedInput

* update forkserver example after merge

* clippy fixes

* fix inproc test

* fixes for cargo hack --feature-powerset

* fix baby_no_std

* implement Hash for NautilusInput

* update fork executor baby fuzzer

* fix doc

* implement Hash for PacketData

* fix windows build

* fix windows no_std

* fix backtrace baby fuzzers README

* add comments

* move setup_bt_panic to constructor

* pre/post child exec hooks in Observer

* setup_child_panic_hook

* fix ObserversOwnedMap on nightly

* add backtrace fuzzers to CI checks

* fix typo

* fix relative paths in test_all_fuzzers.sh

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-01-31 15:58:15 +01:00
Evan Richter
4a6616bdfe
[libafl_qemu] simplify emu::{read,write}_mem (#496)
Methods read_mem and write_mem now operate on &[u8], not &[T]

The generic T slice interface was prone to various footguns:
* i32 is the default Rust integer type, but buffers are often expected
  to hold u8. This means the following code writes 16 bytes to the
  guest, not 4:

      let buf = [0; 4];
      emu.write_mem(addr, &buf);

* If a buffer of 16-bit or larger integers (&[u64] for example) is
  needed to read/write, the user will need to consider host/guest
  endianness. The byte array methods in std are a good, explicit
  alternative.

  Perhaps libafl_qemu could expose/define "to/from guest endianness"
  helper functions or extension traits using the established cfg flags,
  so that guest endianness is always right by default.

* emu::read_mem causes insta-UB if a user did something like:

      let mut my_bool = false;
      emu.read_mem(addr, &mut my_bool);

  It's less surprising for users to just operate on plain-ol' bytes,
  which they can explicitly transmute if they wish.
2022-01-27 09:05:33 +01:00
Dominik Maier
77e5965e97
Add AsSlice, AsMutSlice traits, refactor MapObservers to be iterable, and have associated types (#477)
* from warning

* fix latest clippy

* clippy fixes++

* renamed shmem parameters

* renamed map to shmem

* make forkserver executor work for any (non-system) shmem

* Mem -> ShMem

* rework windows

* fix nit

* fix symbolic

* refacctor map observers

* iterator for map observers

* removed unused ownedptr, added asslice trait to most functions

* make map entry type an associated type

* fix fuzzers

* fix docs

* typo fix

* fix windows, add try_from_slice to shmid

* missing import

* fix fuzzbench

* cleanup

* fmt

* more asslice

* fmt

* added doc link about token-level fuzzing

* cods
2022-01-19 00:02:33 +01:00
Andrea Fioraldi
bca1f392a7
Bump to 0.7.1 (#465)
* bump to 0.7.1

* bump libafl_qemu
2022-01-13 11:32:57 +01:00
Andrea Fioraldi
6274ad4594
Refactor libafl_qemu creating the Emulator struct and post syscall hooks (#430)
* working without asan.rs

* working asan

* update fuzzers

* mremap in snapshot

* sugar

* python

* fix python

* clippy

* fmt

* fuck you loader
2021-12-23 09:10:13 +01:00
Dominik Maier
88e07a8d37 CI galore 2021-12-15 23:34:42 +01:00
Dominik Maier
83583a867f
QEMU target arch selector via feature flag (#405)
* QEMU target arch selector via feature flag

* fix ci

* fixing ci some mmore

* more ci fixes, defaulting to x86_64 always

* more ci

* i368 -> i386 typo fix

* revert forkserver changes

* trying to fix clippy

* docs

* fixed warnings

* more clippy action

* qemu example arch

* bring back deprecated function I don't know how to replace

* get rid of deprecated feature again'

* builds?i
2021-12-06 20:06:47 +01:00
Dominik Maier
a0ce4cfd68
Ignored qemu fuzzer for non-linux (#397)
* ignored qemu fuzzer for non-linux

* fixed cfg

* ignore rm -rf errors in make short_test (fuck you macos)

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2021-12-02 11:48:35 +01:00
Andrea Fioraldi
37b8cb0d2f Bump to 0.7 2021-12-01 17:22:09 +01:00