88 Commits

Author SHA1 Message Date
Addison Crump
3d702f403b
Implement CanTrack tracking enforcement through rust types (#1886)
* sample implementation of tracking enforcement (incomplete)

* helpful compiler output

* make it look like a real compiler output

* ensure that the macro may be used outside of libafl

* separate index/novelty tracking funcs

* default const generic values so that we don't need to change this everywhere

* fix tests

* rollback unnecessary specification of stdmapobserver

* register metadata in doc tests

* doc fixes

* doc cleanup

* doc cleanup 2

* reduce implementor overhead to zero

* renaming/docs fixes

* asref isn't reflexive??

* generalization stage updates

* add better documentation about require_{indices,novelties}_tracking

* remaining generic updates

* round one CI pass (knowingly introduces breaking changes)

* typo

* round 2 clippy

* rollback: libafl_frida changes

* fmt

* moar porting

* fix remaining fuzzers

* fix windows build, maybe

* fixup libafl_libfuzzer

* fmt nighlty all the things

* attempt to fix some broken additions

* fix fmt

* oops

* fix new invocation

* minimizer scheduler fixes

* fix accounting

* rename

* fix

* Fix build

* Sort generics

* Move more generics into the right place

* Rename A -> C

* Fix test

* Fix test some more

* Fix doc some more

* critical formatting

* More A->C

* CanTrack harder

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-04-12 18:33:52 +02:00
Dominik Maier
ff938261df
Fix Clippy on MacOS for qemu, accept some clippy::missing_transmute_annotations suggestions (#2009)
* Fix Clippy/CI

* Ignore transmute annotations in pybind

* fix

* more clippy

* more clippy

* fix fixes
2024-04-08 16:24:10 +02:00
Dongjia "toka" Zhang
527b892c1d
Separate fuzzer test in CI (#2010)
* ci

* ci

* ci

* aa

* aa

* a

* a

* a

* fix

* no fail fast

* system

* what is this lint...

* a

* clp

* disk

* ok no fastfail

* no qemu for now, reenable monday
2024-04-06 21:01:57 +02:00
Dominik Maier
a0143d02a6
Rename OSError -> OsError and merge with redundant Error::File (#1944)
* OSError -> OsError

* Move File errors to io Errors

* Removing File errors

* fixes :)

* More format

* fix libfuzzer runtime
2024-03-18 08:46:48 +01:00
Dongjia "toka" Zhang
c6420c0987
Separate fuzzbench fuzzers's test (#1947)
* separate fuzzbench test

* upd
2024-03-15 19:44:22 +01:00
Dongjia "toka" Zhang
001e132186
Fix windows after #1350 (#1801)
* fix

* apt update

* apt update

* a

* split even more

* tmate

* ci

* can't fix this shit

* s
2024-01-17 18:51:56 +01:00
Dominik Maier
88f3ff4a28
Clang-format latest (#1651)
* Clang-format latest

* add

* rdm

* CHATGPT

* CHATDUMB

* different action

* no

* fix

* ci

* ci

* ci

* fuck

* dir

* checkout?

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* don't use shit

* ci

* ci

* ci

* FMT

---------

Co-authored-by: toka <tokazerkje@outlook.com>
2024-01-05 23:32:06 +01:00
Dongjia "toka" Zhang
0a72d8abbc
Add header (#1777)
* add

* rdm

* CHATGPT

* CHATDUMB

* different action

* no

* fix

* ci

* ci

* ci

* fuck

* dir

* checkout?

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* ci

* don't use shit

* ci

* ci

* ci
2024-01-05 19:35:36 +01:00
Dongjia "toka" Zhang
fad59987d9
Better SigInt handler (#1710)
* fix simd

* better exit

* chg

* more

* more

* use

* change to proper name

* w

* aaaaaaa

* delete

* just separate them

* shell check
2023-12-04 20:06:11 +01:00
Dominik Maier
8545778780
Revert "Use clang-format-16 (#1649)" (#1650)
This reverts commit 1aede04af75564d4063f91d1c8c1a402760e86f5.
2023-11-04 21:38:36 +01:00
Dongjia "toka" Zhang
1aede04af7
Use clang-format-16 (#1649) 2023-11-04 17:45:01 +01:00
Dongjia "toka" Zhang
a9014a9419
fmt c too (#1593)
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-10-03 11:43:28 +02:00
Dongjia "toka" Zhang
324db072a2
Fix CI (#1558)
* endif

* just fix every profile

* WHY YOU DONT JUST USE THE SAME NAME FOR DIR

* fix

* Some fuzzer want release

* WHY I ALWAYS FORGET COMMA

* NO MORE SPACE

* rename

* stb doesn't like debug build

* just use release

* another just use release
2023-09-27 09:59:55 +02:00
Dongjia "toka" Zhang
c1006c4d38
Fix CI (#1553)
* export PROFILE

* install aflplusplusw

* no dot

* a
2023-09-25 20:51:40 +02:00
Dongjia "toka" Zhang
c1eef33b5d
Remove --release from Makefile.toml (#1522)
* rem release

* Revert "rem release"

This reverts commit 13aaf13e412818629880348188c3e1b4137b9ed8.

* toggle it with envvar

* fm
2023-09-25 14:22:10 +02:00
Addison Crump
8f6efe993d
Address recent clippy changes; build clippy in debug instead of release for perf (#1516) 2023-09-19 00:42:46 +02:00
Andrea Fioraldi
e66eb33e96
Fix libafl_libfuzzer publish (#1475)
* fixup build

* allow dirty

---------

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2023-08-29 11:32:52 +02:00
Andrea Fioraldi
062ae9d544
Fix doc for publish (#1472)
* Fix doc for publish

* add bolts to publish.sh
2023-08-28 17:22:44 +02:00
Addison Crump
9aa40c0734
Document libafl_libfuzzer (#1457)
* prep for publishing libafl_libfuzzer

* learn to use linkers

* document-features

* special handling for fuzzbench builds

* Update cmplog.c

* drop dep for llvm-tools; add testcase for memcmp sanity

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-08-24 20:24:38 +02:00
Addison Crump
862de53cf6
Full libfuzzer shimming (for cargo-fuzz libfuzzer alternative and other use cases) (#981)
* squash libfuzzer edits

* fixup: compat with custom mutators

* use tui flag

* add introspection support

* use libfuzzer dep now that we've merged

* force input loading

* some fixes

* begin docs, impl shrink

* make whole-archive conditional and not default

* make more copies of counters maps

* lol, remember to add the observer

* make size edge map observer an observer

* fixup: make def of run driver conditional

* add sanity checks for insertion

* revert silencing of forks

* add experimental tmin support; add default asan flags

* use default options instead of specifying our own

* implement lockless mode

* fix merge

* fixup lockless corpus

* fixup for generalisation

* remove erroneous drop_in_place

* improve error logging in the case of corpus loading failure

* ok, use lock files 😔

* fix tmin

* implement merge (again); fix rare cases with maps being too small

* implement a scheduler for removing excess

* implement a walking strategy for corpus loading for large corpora

* revert filename parameter; rename and remove duplicates

* various cleanup and clippy satisfaction

* fix no_std tests

* clang-format

* expand and satisfy the clippy gods

* fix sanitizer_ifaces bindgen for no_std

* fix wasm fuzzer

* fixup clippy script

* rename and provide a small amount of explanation for sanitizer_interfaces

* fixup: HasLastReportTime

* fix clippy oddities

* restrict clippy checks to linux-only for libafl_libfuzzer_runtime

* name the mutators

* format

* fix clippy warning

* hope docker is fixed

* fix cmin lint

* clippy pass

* more docs

* more clippy

* fix remaining clippy complaints

* fix import

* miri fixes (no constructors executed)

* exclude libafl_libfuzzer from cargo-hack

* fix clippy check for sanitizer_interfaces

* fmt

* fix CI (?)

* deduplicate sancov 8bit for improved perf on ASAN

* merge 8bit coverage regions + comment out insane deduplication

* no erroring out on free hooks

* fixup for non-forking merge

* skip the corpus dir if we use it

* fixup: recent libafl changes and feature flags

* libafl_libfuzzer: use rust-lld for whole-archive feature

* clarify cause of failure

* mark unsafe

* clippy :cursed_cowboy:

* attempt to fix wasm

* spooky unknowable bug 👻

* more clippy lints

* clippy fix for merge

* use the version pin

* add unsafe to ::register

* Serdeany autoreg fix

* make type assert actionable

* miri fixes

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
Co-authored-by: Mrmaxmeier <Mrmaxmeier@gmail.com>
2023-08-24 13:30:23 +02:00
Andrea Fioraldi
0b43711dc9
Fix LLMP p2p + restart bug with CentralizedEventManager (#1389)
This commit rewrites the centralized manager to use a secondary broker, as p2p communication is unreliable during frequent restarts. A centralized launcher is introduced too.
2023-08-22 15:57:50 +02:00
Andrea Fioraldi
258780370f
Update publish.sh (#1294) 2023-05-24 14:16:09 +02:00
Dominik Maier
20f8cb10eb
Revert "Automatically add the comment about executions when a new PR triggers (#1270)" (#1271)
This reverts commit d2e4b9f206307db0898caac707c0d2687079bfb1.
2023-05-17 02:48:15 +02:00
ToSeven
d2e4b9f206
Automatically add the comment about executions when a new PR triggers (#1270)
Co-authored-by: toseven <Byone.heng@gmail.com>
2023-05-17 00:30:15 +02:00
ToSeven
659e91fb68
Fix performance regression detection in CI #1248 (#1259)
Co-authored-by: toseven <Byone.heng@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-05-14 12:02:32 +02:00
Dongjia "toka" Zhang
721c02cd2c
Revert "Add a CI task that checks performance regression (#1248)" (#1254)
This reverts commit 6929c89b861faa4f382a51ab07e3ee1eb727df8c.
2023-05-04 20:03:06 +02:00
ToSeven
6929c89b86
Add a CI task that checks performance regression (#1248)
Co-authored-by: hengzh <byone.heng@gmail.com>
2023-05-04 11:51:42 +02:00
Andrea Fioraldi
fdf579bcd5
Bump to 0.9.0 (#946)
* bump to 0.9.0

* fix libafl_tinyinst

* fix

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-01-31 10:42:01 +01:00
Dongjia "toka" Zhang
9458549fef
Remove declare -A (#976)
* fix

* fix

* debug

* debug

* older version

* newer version

* fix

* unix?

* fix
2022-12-30 02:02:38 +09:00
Andrea Fioraldi
55e220f0e8
CI: diff with main and HEAD^ (#945) 2022-12-14 01:10:36 +01:00
Andrea Fioraldi
162de0ceaf
Fix CI diffing (#944)
* fix build_and_test_fuzzers to diff from origin/main

* fix fuzzbench_forkserver
2022-12-13 21:19:44 +01:00
Andrea Fioraldi
08be5f732e
CI: Only test fuzzers with diffing deps (#940)
* build and test fuzzer crate

* diffing fuzzers ci only

* clippy

* clippy merda

* clippy merde

* improve it

* comment

* split ubuntu CI workflow

* fix
2022-12-13 14:10:34 +01:00
Dominik Maier
e5aaf85d3c
Tuneable Stage, Scheduler, ScheduledMutator (#874)
* Tuneable Stage, Scheduler, and Mutators

* rename

* fix build

* get rid of prelude

* fmt

* Reworked tunable, fixed stuff, add testcase

* clippy

* further fixes

* fix typo, fmt
2022-11-12 03:02:54 +01:00
Dominik Maier
6b6570ae5f
Use Associated Types for Has* traits and AsSlice (#881)
* More Associated Types

* AsSlice associated-ified

* added script to find unused files

* fixes for python

* build all the things

* windows fixes
2022-11-10 09:31:04 +01:00
Mrmaxmeier
8e6eaf7002
check_for_blobs.sh: respect gitignore (#876) 2022-11-05 23:30:26 +01:00
Andrea Fioraldi
40269a578b
Delete blob and add CI check (#867)
* Delete blob and add CI check

* fix

* shellcheck
2022-10-27 16:20:29 +02:00
Mrmaxmeier
64bc5d5bdb
CI: speedups and fixes (#855)
* scripts/clippy.sh: remove cargo clean step

Clippy used to only report warnings in code that was not part of the
incremental cache. This has changed since and I believe we can safely
drop the `cargo clean` step.

* Revert "ci: install z3 to avoid building from source"

This reverts commit 6ff1c4088811040dcfdbd12273f0baf507a4308b.
This doesn't do anything as we're using `static-link-z3` explicitly.

* refactor test_all_fuzzers

nyx tests were not included in `time_record` before this

* nyx fuzzer: move modprobe logic out of setup_libxml2.sh
2022-10-23 01:58:33 +02:00
Mrmaxmeier
cedcee01c0
CI: Build fuzzers with shared cargo target dir (#845)
* build fuzzers with shared cargo target dir

* Make external build scripts aware of CARGO_TARGET_DIR

* fix libmozjpeg fuzzer with shared target dir

* fix cargo-make default value for CARGO_TARGET_DIR

* avoid ./ in cargo-make for windows compat

* CI: cargo-hack's --feature-powerset is too powerful

* fuzzer_concolic: support CARGO_TARGET_DIR

* ci: install z3 to avoid building from source

* ci: update actions

* ci: test nightly features with nightly rust

* test_all_fuzzers: try pruning more compilation artifacts

* ci: fix nightly feature check

* ci: apply rust-cache action after checkout (d'oh)

The rust-cache action populates the checkout directory, which is promply
deleted by the checkout action during checkout.. whoops!
2022-10-20 21:38:58 +02:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
syheliel
758e49ac70
NYX Executor (GSoC '22) (#693)
* Add ccache

* Update codecov.yml

* Add libnyx

* Fix

* Add nyx build script

* Fix build.sh && init executor.rs

* Fix commit

* Fix code

* initialize `exector.rs`

* refine API in `nyx_bridge.rs`

* initialze `run_target`

* add `test_nyxhelper`

* initize `test_executor`

* remove `nyx_beidge.rs`

* make `test_executor` compile

* Improve test

* refine code

* update version

* fix docker

* fix docker

* Fix clippy

* Fix build

* fix build && add `set_timeout`

* Fix and refine CI

* fix CI

* Fix CI

* Add platform restrict

* cargo fmt

* add parallel mode

* add example `nyx_libxml2_parallel`

* fix fuzzer example

* fix CI

* add README

* fix CI

* fix CI

* fix CI

* remove unwrap and NyxResult

* code format fix

* add libnyx's rev

* fix format

* change Duration format && Fix CI

* caego fmt

* fix CI

* fix CI

* Add doc

* test CI

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* Update test_all_fuzzers.sh

* add cache for apt and cargo-install

* Update build_and_test.yml

* Update build_and_test.yml

* tmp test CI

* fix CI

* remove debug cmd

* remove test

* code refine

* code refine

* code refine

* code refine

* add Makefile

* fix example doc for nyx

* add `NyxHelper::new_with_initial_timeout`

* fix `NyxHelper::new`

* fix curl parameter

* code refine

* add check for setup script

* use afl-clang-fast in nyx

* fix logic

* fix makefile

* fix CI

* Update build_and_test.yml

* Update build_and_test.yml

* remove debug cmd

Co-authored-by: syheliel <syheliel@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2022-08-25 10:55:35 +02:00
Dominik Maier
5ad6519456
Use nightly fmt (#728) 2022-08-14 11:00:02 +02:00
Dongjia Zhang
8899a3b01c
Use clang-format-13 (#705) 2022-07-18 22:31:01 +02:00
Dongjia Zhang
999eaadc16
fix (#703) 2022-07-17 21:15:45 +02:00
Dongjia Zhang
405a1919b8
Still fixing ci (#683)
* fix

* fix

* more

* cargo fixed??

* fixed??

* clp

* fix
2022-06-28 16:56:28 +09:00
Dongjia Zhang
fd9d126124
Fix CI (#681)
* Update test_all_fuzzers.sh

* --

* fix ui.rs

* fix

Co-authored-by: Dominik Maier <dmnk@google.com>
2022-06-27 23:19:15 +09:00
Dominik Maier
bfe69aea09
Format C/Cpp code in ./scripts/fmt_all.sh (#653)
* format all (clang format 13)

* added clang-format check

* re-add missing newline

* cargo doc was missing

* more brackets

* fixed fmt workflow

* clang format

* shellcheck

* install clang-format-13

* update ubuntu for maximum clang-formattability

* yml whitespaces

* fmt

* shellcheck only for .sh

* oops path

* ignored shellcheck warning
2022-05-29 03:23:02 +02:00
Dominik Maier
e4447364c2
SymCC update (#656) 2022-05-28 01:41:43 +02:00
Dominik Maier
828ebcff39
Clippy nits & fixes (#640)
* release autofix

* fix unused backtrace

* clippy fixes

* clippy

* more clippy

* more autofix

* clippy for frida

* more clippy
2022-05-22 02:43:25 +02:00
Dominik Maier
50ddbf6a78
Added autofix script (#639) 2022-05-21 15:06:11 +02:00
syheliel
2ead2c398e
Speed up CI (#630)
* Add ccache

* Update codecov.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update test_all_fuzzers.sh

Add fuzzer timer

* Fix `./test_all_fuzzers.sh` on macos

* Fix CI

* Fix CI

* Update build_and_test.yml

* Fix typo

* Set mold linker as default linker

* Fix CI

* Update build_and_test.yml

* Add profile arguments

* Fix CI

* Update test_all_fuzzers.sh

Co-authored-by: syheliel <syheliel@gmail.com>
2022-05-15 21:25:29 +02:00