100 Commits

Author SHA1 Message Date
Dominik Maier
20f8cb10eb
Revert "Automatically add the comment about executions when a new PR triggers (#1270)" (#1271)
This reverts commit d2e4b9f206307db0898caac707c0d2687079bfb1.
2023-05-17 02:48:15 +02:00
ToSeven
d2e4b9f206
Automatically add the comment about executions when a new PR triggers (#1270)
Co-authored-by: toseven <Byone.heng@gmail.com>
2023-05-17 00:30:15 +02:00
ToSeven
659e91fb68
Fix performance regression detection in CI #1248 (#1259)
Co-authored-by: toseven <Byone.heng@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-05-14 12:02:32 +02:00
Kevin Phoenix
be1d3da159
Add pyproject.toml to python bindings (#1239)
* Add pyproject.toml to python bindings

* Improve pyproject.toml

* Update CI pipeline to use pip to install python bindings
2023-05-09 13:23:28 +02:00
Dongjia "toka" Zhang
721c02cd2c
Revert "Add a CI task that checks performance regression (#1248)" (#1254)
This reverts commit 6929c89b861faa4f382a51ab07e3ee1eb727df8c.
2023-05-04 20:03:06 +02:00
ToSeven
6929c89b86
Add a CI task that checks performance regression (#1248)
Co-authored-by: hengzh <byone.heng@gmail.com>
2023-05-04 11:51:42 +02:00
Dominik Maier
f9c74ed5d6
Update llvm for FreeBSD CI (#1243) 2023-05-03 11:45:49 +02:00
Dominik Maier
3f7d35bfdc
Install libz3-dev in CI (#1163)
* install z3 in CI

* Update logics.rs

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-03-22 15:38:33 +01:00
Dominik Maier
f4f23de32b
Fix fuzzers after HasTestcase (#1123) (#1162)
* Fix fuzzers after HasTestcase (#1123)

* Make the trait a trait

* Implement HasTestcase for Corpora

* fix

* fix

* a

* a

* fix

* wasm32

* a

* f

* f

* aa

---------

Co-authored-by: tokatoka <tokazerkje@outlook.com>
2023-03-19 11:58:32 +09:00
Dominik Maier
306cdcd800
Frida: Fix Android build (#1154)
* update android version

* fix android build some more
2023-03-16 16:26:22 +01:00
Dominik Maier
2ed6583041
CI: Run miri tests (#1130)
* Fixes/ignores for miri support

* linux

* fix doctest for miri

* fix docs

* fix UB in baby_fuzzer

* no custom allocator in miri
2023-03-08 19:21:17 +01:00
Addison Crump
0727c80347
Add example for WASM (#1093)
* add baby_fuzzer for wasm targets

* elaborate in README

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-02-26 17:02:22 +01:00
Dongjia "toka" Zhang
95004aab7e
Update build_and_test.yml (#1103) 2023-02-25 15:12:59 +09:00
Fabian Freyer
53dba5f49d
Use the log facade instead of println (#1060)
* switched a couple of println / dbg statements to use log crate
* Use pyo3-log for logging in python bindings
2023-02-14 10:01:51 +01:00
Dominik Maier
d73fb92ddf
Python CI (#1024)
* Python CI

* fix testcase

* fix yml

* Fixing test

* format python

* cleanup
2023-01-31 05:04:19 +01:00
Dongjia "toka" Zhang
fe51286586
TinyInst Update (#968)
* tmp

* more

* save

* TODO

* fix

* update to tinyinst on crates

* dep

* fmt

* shmem done

* cpp fmt

* clp

* fmt

* why??

* ver

* more makefile.toml

* windows test

* Update build_and_test.yml

* fix

* a

* install

* fmt

* fix

* only macos and win

* more

* The order matters

* remove

* fmt

* chg

* typo

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2023-01-23 11:11:53 +01:00
Andrea Fioraldi
7fd9ac0952
SyncFromBrokerStage to sync from a broker with a different Input type (#997)
* ConverterLlmpEventManager

* docs

* SyncFromBrokerStage

* fix

* separate InputConverter

* nautilus

* nautilus_sync

* send testcases

* upd nautilus

* meta

* fix

* clippy

* fix

* Update build_and_test.yml

* fix

* fix

* Use find_libpython

* ci

* upd qemu
2023-01-20 17:16:29 +01:00
Dominik Maier
159e6ea480
Fix Launcher for M1, fix frida_libpng harness compilation, fix CI (#987)
* Fix harness compilation for frida_libpng

* M1: Always use fast cores

* always ignore result

* seeing if manualy installing libunistring fixes wget

* seeing if manualy installing wget fixes it, instead

* un-remove comment
2023-01-05 14:26:20 +01:00
Dongjia "toka" Zhang
476cb7e7dc
Frida Makefile.toml fix (#969)
* frida build script fix

* fix

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml
2022-12-26 14:19:37 +09:00
Dominik Maier
75f12bd0eb
Remodelling Observers/Examples that rely on UB, API cleanups (#950)
* Tackling UB

* PtrMut -> MutPtr, moved mapobservers to non-UB

* QEMU fixes

* test fixes

* qemu

* Change all interfaces, fix all fuzzers

* fixes

* fix more fixes

* fmt

* fix qemu sugar

* fix some qemus

* atheris

* fmt

* more fmt

* most fmt

* more fix

* nyx fyx

* fix qemu

* clippy, fixes

* more fixes

* no unfix, only fix

* fix

* fix

* more clippy

* fixes

* ListObserver

* fmt, clippy

* fix qemu on arm

* update zlib target

* fix?

* fix

* added migration guide

* ignore doc

* fix symcc

* fix new win fuzzer

* Fixes, rename PTR_SIZE to PTR_NUM

* Try fix linking on win

* Trying to fix win linking

* more cov

* trying to fix win some more

* trying to fix mac

* trying to fix mac

* Fix tests

* Fix tests

* trying to fix win

* more mac

* giving up for windows

* fmt

* python3

* mac?

* undo windows tests
2022-12-24 14:20:44 +01:00
Dongjia "toka" Zhang
038732bd92
no mold (#964) 2022-12-21 13:27:14 +09:00
Dongjia "toka" Zhang
f1b25fed65
Fix CI (#955)
* fix

* fmt

* Update build_and_test.yml

* no nightly

* fix

* why

* Update build_and_test.yml

* a

* a

* ok

* linux only

* fmt

* Update build_and_test.yml

* fix
2022-12-21 11:12:54 +09:00
Andrea Fioraldi
162de0ceaf
Fix CI diffing (#944)
* fix build_and_test_fuzzers to diff from origin/main

* fix fuzzbench_forkserver
2022-12-13 21:19:44 +01:00
Andrea Fioraldi
08be5f732e
CI: Only test fuzzers with diffing deps (#940)
* build and test fuzzer crate

* diffing fuzzers ci only

* clippy

* clippy merda

* clippy merde

* improve it

* comment

* split ubuntu CI workflow

* fix
2022-12-13 14:10:34 +01:00
Andrea Fioraldi
c2776e117a
emu::current_cpu() is now the CPU that hitted the breakpoint in fullsystem (#910)
* emu::current_cpu() is now kept after vm stop and it is the CPU that hitted the breakpoint

* clippy

* uninit

* clippy

* clippy

* clippy

* clippy

* nightly override in CI

* nightly override in CI

* components

* components

* targets

* targets

* clippy

* clippy

* clippy

* clippy

* clippy (again)

* MaybeUninit

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-25 11:57:08 +01:00
Andrea Fioraldi
7b0039606b
Forksrv adaptive map size and AFL++ CmpLog support (#896)
* AFL++ cmplog map

* map size opt in forkserver

* MapObserver::downsize_map and adaptive map size in forkserver

* fix fokserver_simple cmd opts

* clippy

* fuzzbench forkserver with cmplog

* delete makefile in fuzzbench forkserver

* fuzzbench_forkserver is persistent

* ForkserverExecutorBuilder::build_dynamic_map

* fix

* clippy

* fix

* fix macos

* fix compilation

* fix bugs

* fixes

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-11-22 10:33:15 +01:00
Alwin Berger
b33839708e
Fix QEMU systemmode fuzzing (#883)
* libafl_qemu: fix systemmode with slirp dependency

libslirp will be dropped from future QEMU releases (see https://wiki.qemu.org/ChangeLog/7.0).
This change adds the "slirp" feature,
which links with the host-systems libslirp.

* libafl_qemu: enable systemmode snapshots, vm_start

Re-enable snapshot functions.
Start the VM before qemu_main_loop.

* libafl_qemu: allow synchronous snapshotting

Add a flag to take snapshots synchronosly.
This should be used to take or load snapshots while the emulator is not
running.

* libafl_qemu: fallback cpu for read-/write_mem

In systemmode, current_cpu may not be set.
In such cases use the first cpus memory access methods.

* fuzzers: add example for libafl_qemu in systemmode

* libafl_qemu: update libafl-qemu-bridge revision

* libafl_qemu: add memory access by physcial address

* fix liabfl_qemu example

Use GuestAddr and physical memory access

* ci: install libslirp-dev for libafl_qemu

* fuzzers/qemu_systemmode: clean up example

* libafl_qemu: remove obsolete functions

emu::libafl_cpu_thread_fn
emu::libafl_start_vcpu
emu::start

* fuzzers/qemu_systemmode: simplify example

* improve build_linux.rs

* Update qemu_systemmode fuzzer

* upd

* clippy

Co-authored-by: Alwin Berger <alwin.berger@tu-dortmund.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-11-21 17:57:06 +01:00
Dongjia "toka" Zhang
ec38858b2d
Fix Makefile.toml (#893)
* don't use submodules

* fix

* add

* fix

* a

* fix

* doesn't work 😩

* fix

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml
2022-11-17 04:44:26 +09:00
Dominik Maier
ff945d9657
Fix CI again (#872)
* Fix CI yml

* Fix CI
2022-10-29 00:05:28 +02:00
Dominik Maier
17d0795f0d
Fix CI yml (#871) 2022-10-28 23:50:38 +02:00
Andrea Fioraldi
40269a578b
Delete blob and add CI check (#867)
* Delete blob and add CI check

* fix

* shellcheck
2022-10-27 16:20:29 +02:00
Patrick Gersch
cf9c4188c0
Disabling qemu dependecies for qemu fullsystem (#737)
* Disabling qemu dependecies by default

* Adding full emulation_mode support

* Removing usermode from libafl_qemu default features

* Fixing refactoring

* Fixing typo in systemmode

* Fixing clippy:needless-borrow

* Mark libafl_load/save_qemu_snapshot as unused + cpu_reset

* Fixing clippy::needless-borrow

* Fixing needless-borrow yet again

* reset_cpu -> cpu_reset

* Fixing github workflow yet again

* Fixing clippy::uninlined-format-args

* Adding current libafl_qemu_bridge

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2022-10-25 14:16:11 +02:00
Andrea Fioraldi
5571a03641
Implement thread-safe AsanGiovese in Rust with snapshots support (#851)
* Purge C impl of asan-giovese

* Compiling

* reset asan

* Restore asan state in qemu

* clippy

* upd

* Asan snapshots

* fuzzbench_qemu

* fix snap mmap limit

* fix

* compiles again

* clippy

* update meminterval

* autofix

* fix 32 bit targets

* try to clean intermediate builds

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-10-25 09:48:59 +02:00
Mrmaxmeier
64bc5d5bdb
CI: speedups and fixes (#855)
* scripts/clippy.sh: remove cargo clean step

Clippy used to only report warnings in code that was not part of the
incremental cache. This has changed since and I believe we can safely
drop the `cargo clean` step.

* Revert "ci: install z3 to avoid building from source"

This reverts commit 6ff1c4088811040dcfdbd12273f0baf507a4308b.
This doesn't do anything as we're using `static-link-z3` explicitly.

* refactor test_all_fuzzers

nyx tests were not included in `time_record` before this

* nyx fuzzer: move modprobe logic out of setup_libxml2.sh
2022-10-23 01:58:33 +02:00
Mrmaxmeier
cedcee01c0
CI: Build fuzzers with shared cargo target dir (#845)
* build fuzzers with shared cargo target dir

* Make external build scripts aware of CARGO_TARGET_DIR

* fix libmozjpeg fuzzer with shared target dir

* fix cargo-make default value for CARGO_TARGET_DIR

* avoid ./ in cargo-make for windows compat

* CI: cargo-hack's --feature-powerset is too powerful

* fuzzer_concolic: support CARGO_TARGET_DIR

* ci: install z3 to avoid building from source

* ci: update actions

* ci: test nightly features with nightly rust

* test_all_fuzzers: try pruning more compilation artifacts

* ci: fix nightly feature check

* ci: apply rust-cache action after checkout (d'oh)

The rust-cache action populates the checkout directory, which is promply
deleted by the checkout action during checkout.. whoops!
2022-10-20 21:38:58 +02:00
Dominik Maier
41cc717dfc
Install no_std nightly toolchain for CI (#847) 2022-10-19 18:19:27 +02:00
Dongjia "toka" Zhang
00227e8058
Fix CI (#817)
* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml
2022-10-05 22:18:50 +02:00
Dongjia "toka" Zhang
e4f0e1df99
Fix CI (#810)
* fix

* fix

* fix

* removed allow(unused)

* removed feature from powerset

* fixing win

* tidying up

* cfg guards galore

* cfg for unused

* more cfg

* more fixes

* more cfgs

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-10-04 17:29:42 +02:00
expend20
ae400e5ce8
Remove clang download from windows CI (#791)
* Attempt to remove clang 12 setup

* frida_gdiplus added to CI
2022-09-19 09:20:33 +02:00
Vincent
10f5c0f07a
Fix cargo doc on windows (#762)
* add doc cfg

* fix nostd docs

* ignore CommandConfigurator doc test execution on non-unix platform

* add cargo doc step pipeline on windows platform
2022-09-04 09:34:26 +02:00
Dominik Maier
c4e0faabc2
CI for FreeBSD (#754)
* CI for FreeBSD

* rustup -y?

* fixed path, switched to clippy

* bsd don't source

* added llvm

* clippy

* more yml

* ?

* testing ci

* llvm?

* llvm??

* more llvm, more tests

* fixed testcase'

* mem limits

* more sudo

* reenable all the CI
2022-09-02 15:37:49 +02:00
syheliel
758e49ac70
NYX Executor (GSoC '22) (#693)
* Add ccache

* Update codecov.yml

* Add libnyx

* Fix

* Add nyx build script

* Fix build.sh && init executor.rs

* Fix commit

* Fix code

* initialize `exector.rs`

* refine API in `nyx_bridge.rs`

* initialze `run_target`

* add `test_nyxhelper`

* initize `test_executor`

* remove `nyx_beidge.rs`

* make `test_executor` compile

* Improve test

* refine code

* update version

* fix docker

* fix docker

* Fix clippy

* Fix build

* fix build && add `set_timeout`

* Fix and refine CI

* fix CI

* Fix CI

* Add platform restrict

* cargo fmt

* add parallel mode

* add example `nyx_libxml2_parallel`

* fix fuzzer example

* fix CI

* add README

* fix CI

* fix CI

* fix CI

* remove unwrap and NyxResult

* code format fix

* add libnyx's rev

* fix format

* change Duration format && Fix CI

* caego fmt

* fix CI

* fix CI

* Add doc

* test CI

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* add cache for apt and cargo-install

* Update build_and_test.yml

* Update build_and_test.yml

* tmp test CI

* fix CI

* remove debug cmd

* remove test

* code refine

* code refine

* code refine

* code refine

* add Makefile

* fix example doc for nyx

* add `NyxHelper::new_with_initial_timeout`

* fix `NyxHelper::new`

* fix curl parameter

* code refine

* add check for setup script

* use afl-clang-fast in nyx

* fix logic

* fix makefile

* fix CI

* Update build_and_test.yml

* Update build_and_test.yml

* remove debug cmd

Co-authored-by: syheliel <syheliel@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-08-25 10:55:35 +02:00
Patrick Gersch
b2a1e03703
Qemu arm launcher (#708)
* Adding qemu_arm_launcher crate

* Trying to fix qemu arm usermode

* Cargo fmt

* Adding CROSS_CC env

* Remove hardcoded arm-linux-gnueabi-gcc and replace by CROSS_CC

* Adding arm-linux-gnueabi-gcc to github workflows for ubuntu

* Fixing typo in apt install package

* Resetting LR after each fuzzing emulation

* Cargo fmt after merge conflict

* Using GuestAddr

* Compiling, running and running with artificial crash detection

* Adding dependencies for github workflow to cross compile for arm

* Fixing github workflow for ubuntu fuzzer

* arm-linux-binutils for mac in github workflows

* Qemu does not work for mac, no need to compile qemu_arm_launcher harness for it
2022-08-02 11:46:24 +02:00
Dongjia Zhang
bc6a032843
Windows CI for frida (#658)
* harness & makefile.toml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* rename

* no stdout

* Update harness_win.cpp

* Update harness_win.cpp

* Update Makefile.toml

* Update build_and_test.yml

* Update Makefile.toml

* Update Makefile.toml

* fix
2022-05-30 23:06:37 +02:00
Dominik Maier
bfe69aea09
Format C/Cpp code in ./scripts/fmt_all.sh (#653)
* format all (clang format 13)

* added clang-format check

* re-add missing newline

* cargo doc was missing

* more brackets

* fixed fmt workflow

* clang format

* shellcheck

* install clang-format-13

* update ubuntu for maximum clang-formattability

* yml whitespaces

* fmt

* shellcheck only for .sh

* oops path

* ignored shellcheck warning
2022-05-29 03:23:02 +02:00
Dominik Maier
763ed9a3e5
Moved to no_std preamble (#643)
* Moved to no_std preamble

* fixed use

* no_std targets

* derive no_std

* fix yml

* ci

* alf

* gitignore

* fix python build

* import cleanup

* nostd

* linux fix
2022-05-27 01:04:29 +02:00
Andrea Fioraldi
28edbad618
Refactor libafl Python bindings (#632)
* SerdeAny MapFeedbackState

* Fix macro syntax

* alloc

* fix

* Metadata calibrate and map feedback

* metadata feedback states

* compile

* fmt

* Register common generic types

* tests

* sugar

* no_std

* fix book

* alloc

* fix fuzzers

* fix

* fmt

* disable python bindings for libafl

* clippy

* fmt

* fixes

* fmt

* compiling python bindings

* no uaf in python observer

* working python observer, feedback and executor

* mutators

* fmt

* nits

* added autofix script

* clippy

* clippy

* more clippy

* fix

* ignore clippy for deserialization

* newlines

* nits

* fmt

* feedbacks

* generators

* methods

* feedbacks

* pyerr

* fix

* fix

* fmt

* python bindings in CI

* fix

* fix

* fix

* autofix

* clippy

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-05-25 16:56:06 +02:00
syheliel
2ead2c398e
Speed up CI (#630)
* Add ccache

* Update codecov.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update test_all_fuzzers.sh

Add fuzzer timer

* Fix `./test_all_fuzzers.sh` on macos

* Fix CI

* Fix CI

* Update build_and_test.yml

* Fix typo

* Set mold linker as default linker

* Fix CI

* Update build_and_test.yml

* Add profile arguments

* Fix CI

* Update test_all_fuzzers.sh

Co-authored-by: syheliel <syheliel@gmail.com>
2022-05-15 21:25:29 +02:00
syheliel
aa101c396a
Merge ubuntu and macos fuzzer && Record time for ./test_all_fuzzers.sh (#629)
* Add ccache

* Update codecov.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update test_all_fuzzers.sh

Add fuzzer timer

* Fix `./test_all_fuzzers.sh` on macos

* Fix CI

* Fix CI

* Update build_and_test.yml

* Fix typo
2022-05-14 00:57:26 +02:00
Dongjia Zhang
936e2221d1
Cargo-make (#537)
* timeout utility

* example build.toml

* upd

* ci

* Update build_and_test.yml

* Update build_and_test.yml

* rename, qemu_launcher

* libpngs

* fix

* upd

* del

* do_nothing -> unsupported

* rename

* use command

* non qemu fuzzbench

* script.sh

* mroe

* qemu

* fix

* generic

* fix

* fix

* allow 124

* quotes

* fix

* fix

* fix

* stderr to devnull

* chg
2022-02-20 03:32:43 +01:00