140 Commits

Author SHA1 Message Date
Dongjia "toka" Zhang
a98805b4ca
Another attemp to fix CI (#1744)
* fixing?

* more

* delete macos build_and_run fuzzers as its runtime exceeds 360minutes and can't fix it
2023-12-21 04:43:01 +09:00
Dongjia "toka" Zhang
78060ea308
0.11.2 (#1735) 2023-12-18 14:33:14 +01:00
Dongjia "toka" Zhang
7894efe728
Last cleanup after decapstone (#1727)
* Last clenup

* more

* more
2023-12-16 19:51:42 +09:00
Dongjia "toka" Zhang
a0a4dd60bb
Remove capstone from frida [x86_64] (#1720)
* init

* more

* just fixing stuff
2023-12-16 02:39:11 +09:00
Dongjia "toka" Zhang
324db072a2
Fix CI (#1558)
* endif

* just fix every profile

* WHY YOU DONT JUST USE THE SAME NAME FOR DIR

* fix

* Some fuzzer want release

* WHY I ALWAYS FORGET COMMA

* NO MORE SPACE

* rename

* stb doesn't like debug build

* just use release

* another just use release
2023-09-27 09:59:55 +02:00
Dongjia "toka" Zhang
c1eef33b5d
Remove --release from Makefile.toml (#1522)
* rem release

* Revert "rem release"

This reverts commit 13aaf13e412818629880348188c3e1b4137b9ed8.

* toggle it with envvar

* fm
2023-09-25 14:22:10 +02:00
Fabian Freyer
7f0a4f1d7e
libafl_frida: Add FridaInstrumentationHelperBuilder, don't rely on Clap options (#1523)
* impr(frida): Don't keep FuzzerOptions in Helper

Instead, keep the actual values that are needed. This allows us to make
a builder for FridaInstrumentationBuilder in a subsequent commit.

* refactor(frida): Move workaround to separate method

This is just code movement.

* refactor(frida): move transformer initialization

Mostly code movement here, sets up replacing `new` with a builder. The
one exception is the introduction of a lifetime bound on RT, which needs
to outlive the transformer. This could be generic, but there's probably
no reason to introduce an additional lifetime.

However, because of this lifetime introduction, this is _technically_ a
breaking change.

* impr(frida): Pass module map to runtimes

Instead of passing a slice of modules to instrument, and re-building the
modulemap, pass a Ref-counted module map directly to the initialization.

* feat(frida): Builder for InstrumentationHelper

Co-authored-by: Dominik Maier <domenukk@gmail.com>

* impr(frida/alloc): optional options in allocator

Move all the initialization into Default::default with sensible defaults
and override parameters set from options in new.

* impr(frida): remove options from AsanError

The only option AsanError uses is whether to continue on error. Instead
of keeping a whole clone of the options around, just store that single
boolean value.

* impr(frida/asan): Use less FuzzerOptions

* Implement Default::default to get a good default AsanRuntime

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-09-20 11:08:59 +02:00
Andrea Fioraldi
00033426e7
Bump to 0.11.1 (#1491) 2023-08-31 15:07:31 +02:00
Andrea Fioraldi
7dd7c1a485
Bump to 0.11.0 (#1469)
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-08-28 15:36:43 +02:00
Dominik Maier
35fa881ff0
Update frida (#1408)
* Update frida

* fix build

* aarch64

* fix aarch64 buid

* Fix CI

* move to git version of frida

* fix

* Frida frida frida
2023-08-17 17:49:12 +02:00
Dominik Maier
b0179b4498
Update some deps, clippy (#1422)
* More less default

* More clippy

* updated rangemap

* Clean up depencdencies

* Undo accidental remove

* Fix

* trying to fix qemu build

* hexagon be gone
2023-08-17 17:15:03 +02:00
Dominik Maier
e9e9c457d6
Move Bolts to libafl_bolts (#1335)
* sort memebers

* Building bolts

* fixing python, feature flags

* Cleanup bolts Cargo.toml

* Fix tests

* cleanup libafl

* removed duplicate examples

* Info text

* reenable agpl CI

* fix impl_serdeany

* new fmt

* Moved bolts

* fix some builds

* fix

* fix more fixes

* serdeany

* no_std

* Dependency cleanup

* Fix docs

* Docker

* add python bolts bindings

* no_std test fix

* merge fail

* typo fix

* add bolts dependency to fuzzers

* tiny fixes

* merge fun

* clippy

* link no longer exists

* make sure python gets rebuilt

* fix pybind

* doc fix

* remove bolts ref

* LibAFL bolts

* More info

* deprecation notice for launcher

* fix python

* cargo fmt

* fix concolic

* fix

* clippy

* fix libafl_cc

* fix tutorial, clippy

* fix concolic fuzzer

* fix push_stage_harness fuzzer

* prelude

* fix testcase post-merge

* mute clippy
2023-08-02 17:36:26 +02:00
Dongjia "toka" Zhang
109755208e
Revert "Make harness function take mut ref (#1338)" (#1358)
This reverts commit fe6daecf0bb178cd19970ae81c797443fd8cd88f.
2023-07-10 17:33:26 +02:00
Rowan Hart
fe6daecf0b
Make harness function take mut ref (#1338)
* Change executor trait to allow \&mut Input

* Add mut inprocess executor

* Add mut inprocess executor

* Format and fix clippy errors

* Fix more clippy errors

* Revert accidental refactoring of InMemoryCorpus

* Add mut versions of all executors that can support it

* Do not persist possible testcase mutation in stages, shadow/differential executors, or corpus minimization

* Fix missing imports

* Fix executor type for missed qemu items

* Add re-exports for mut executors

* Use InProcessForkExecutorMut in QemuForkExecutorMut

* Update BytesInput harnesses to take mutable references

* Update other-input-type-taking harnesses to take mut references

* Clippy fixes

* Feature gate TryFromIntError import

* Fix missed harness input type in baby_fuzzer

* Fix additional clippy issues

* Fix unnecessary hashes on string literal

* Even MORE clippy fixes

* Fix one more clippy issue

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-07-10 13:06:38 +02:00
Dongjia "toka" Zhang
6f21cb3848
Bump to 0.10.1 (#1280)
* bmp

* remove DEBUG env var
2023-05-22 12:42:36 +02:00
Andrea Fioraldi
0f633962ff
Bump to 0.10.0 (#1156)
* Bump to 0.10.0

* fix

* Fix CI

* Fix copyright

* fmt

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-04-05 17:49:53 +02:00
Dongjia "toka" Zhang
ccd1211cd2
Remove qemu_arm_launcher test from CI (#1179)
* remove qemu arm

* trigger all

* debug

* revert

* api chg

* fix

* try

* debug

* remove qemu

* no_std

* Update build_and_test.yml

* llvm16

* revert z3

* macos

* fix

* remove test

* don't grep on mac

* fix

* ok

* f

* f

* f

* aaa
2023-04-04 14:49:58 +02:00
Dominik Maier
bbe4e85768
Removed new_ from constructors that don't need it (API consistency) (#1159)
* Removed new_ from constructors that don't need it (API consistency)

* un-change python bindings
2023-03-17 17:02:21 +01:00
Matheus Baptistella
c38405ef83
Shorthand functions to get typed metadata, renamed metatdata -> metadata_map (#1123)
* Created macro to get the metadata form State and Testcase

* Expanded the macros for mutable, or not, State and Testcase metadata

* Created functions on traits HasMetadata and HasNamedMetadatato get, mutable or not, metadata

* Created the functions to get metadata

* Added #[inline] attribute and renamed the functions

* Renamed the functions and added #[inline] attribute

* Temporarily added testcase() function

* Added testcase() function

* Changed Ref import to core::cell:Ref

* Added testcase_mut() and renamed occurences of metadata() and metadata_mut()

* Renamed more occurences

* Renamed the metadata() on impl HasMetadata for NopState

---------

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
2023-03-14 14:24:33 +01:00
Vincent
c8254dbd0e
Check CI result on cargo make test for available fuzzers (#1107)
* fix libfuzzer_libpng_cmin

* fix libfuzzer_libpng_ctx

* revert libfuzzer_libpng_cmin and check ci by grepping broker stdout result instead

* revert libfuzzer_libpng_ctx and check ci by grepping broker stdout result instead

* add check ci for fuzzers

* add check ci for fuzzers

* add check ci

* add fuzzbench test

* add validation for qemu fuzzer

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2023-03-06 23:21:24 +09:00
Dongjia "toka" Zhang
30b51bb810
Remove unused dependencies (#1069) 2023-02-14 09:02:18 +01:00
Dongjia "toka" Zhang
a74e5da268
Revert FridaInstrumentationHelper changes (#1062)
* Revert "Send stability in calibration stage & FridaInstrumentationHelper retunrs Result<Self, Error> (#1056)"

This reverts commit 4d78878c02846b2c0a49686cd05cccadd2b0ac72.

* fux

* poc

* revert
2023-02-13 10:02:26 +09:00
Dongjia "toka" Zhang
4d78878c02
Send stability in calibration stage & FridaInstrumentationHelper retunrs Result<Self, Error> (#1056)
* fix

* fix

* clippy
2023-02-13 05:35:09 +09:00
R. Elliott Childre
5d76707ede
Bump deps and fix Clippy warns in example fuzzers (#1043)
* Mostly addressing changing the `uninlined_format_args` lint which was
  changed to warn-by-default in rust clippy 1.67

* Bump dependencies:
  bindgen:  0.61 -> 0.63
  cc:       1.0 -> 1.0.42 (Exclue versions w/incompat rayon dependency)
  clap:     3.x -> 4.0
  rangemap: 0.1 -> 1
  xz -> xz2:  move to updated version

* Add fallthrough default return to `LLVMFuzzerTestOneInput` in
  **/fuzz.c to prevent Clang's -Wreturn-type

* libafl_atheris: Improve POSIX compatibility and reduce warnings
  * Check for .dylib and .so libraries
  * `source` -> `.` for POSIX shells
  * install wheel into the venv to support newer Python packaging
    standards
  * `LDPRELOAD` -> `LD_PRELOAD`
2023-02-05 21:53:45 +01:00
Andrea Fioraldi
eaf5ff9de0
Restart loading initial inputs even after a crash/timeout (#1040)
* Track initial inputs loading

* libfuzzer libpng

* fuzzbench

* fix no_std

* fix no_std

* clippy

* fuzzers
2023-02-03 11:56:47 +01:00
Dongjia "toka" Zhang
db62c26eda
Remove unused imports (#1035)
* remove unused imports

* fmt
2023-02-01 12:26:56 +01:00
Andrea Fioraldi
fdf579bcd5
Bump to 0.9.0 (#946)
* bump to 0.9.0

* fix libafl_tinyinst

* fix

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2023-01-31 10:42:01 +01:00
Dominik Maier
97e88af0c5
OnDiskCorpus: Write metadata by default, metadata gzip compression (#995)
* Write metadata by default

* fix fuzzers

* Cleanup, gzip feature

* Fix casing for ondisk corpus

* fix fmt, clippy

* clippy

* clippy for gdiplus fuzzer

* fmt
2023-01-13 01:07:36 +01:00
Dominik Maier
17cb317429
Fix early drop for frida transformer (#992)
* Fix early drop for frida transformer

* clippy
2023-01-08 18:19:30 +01:00
Dominik Maier
159e6ea480
Fix Launcher for M1, fix frida_libpng harness compilation, fix CI (#987)
* Fix harness compilation for frida_libpng

* M1: Always use fast cores

* always ignore result

* seeing if manualy installing libunistring fixes wget

* seeing if manualy installing wget fixes it, instead

* un-remove comment
2023-01-05 14:26:20 +01:00
Dongjia "toka" Zhang
476cb7e7dc
Frida Makefile.toml fix (#969)
* frida build script fix

* fix

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml
2022-12-26 14:19:37 +09:00
Dominik Maier
75f12bd0eb
Remodelling Observers/Examples that rely on UB, API cleanups (#950)
* Tackling UB

* PtrMut -> MutPtr, moved mapobservers to non-UB

* QEMU fixes

* test fixes

* qemu

* Change all interfaces, fix all fuzzers

* fixes

* fix more fixes

* fmt

* fix qemu sugar

* fix some qemus

* atheris

* fmt

* more fmt

* most fmt

* more fix

* nyx fyx

* fix qemu

* clippy, fixes

* more fixes

* no unfix, only fix

* fix

* fix

* more clippy

* fixes

* ListObserver

* fmt, clippy

* fix qemu on arm

* update zlib target

* fix?

* fix

* added migration guide

* ignore doc

* fix symcc

* fix new win fuzzer

* Fixes, rename PTR_SIZE to PTR_NUM

* Try fix linking on win

* Trying to fix win linking

* more cov

* trying to fix win some more

* trying to fix mac

* trying to fix mac

* Fix tests

* Fix tests

* trying to fix win

* more mac

* giving up for windows

* fmt

* python3

* mac?

* undo windows tests
2022-12-24 14:20:44 +01:00
Dongjia "toka" Zhang
ec38858b2d
Fix Makefile.toml (#893)
* don't use submodules

* fix

* add

* fix

* a

* fix

* doesn't work 😩

* fix

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml
2022-11-17 04:44:26 +09:00
Dominik Maier
2011ed299b
Pthread introspection hook (extends #263) (#891)
* Add pthread_introspection_hook support on macos

See-also: #68

* Remove lazy_static

* all of apple are created equal

Co-authored-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
2022-11-15 18:27:48 +01:00
Dominik Maier
e340d35674
Add standalone toolchain link to frida_libpng (#890) 2022-11-11 16:06:38 +01:00
Dominik Maier
663a33168e
Associated types for Corpus, State (#767)
* Associated types for Corpus, State

* cleanup

* fix no_std

* drop unused clauses

* Corpus

* cleanup

* adding things

* fixed fuzzer

* remove phantom data

* python

* progress?

* more more

* oof

* wow it builds?

* python fixes, tests

* fix python fun

* black fmt for python

* clippy, added Nop things

* fixes

* fix merge

* make it compile (#836)

* doc-test fixes, prelude-b-gone for cargo-hack compat

* fixes for windows, concolic

* really fix windows, maybe

* imagine using windows

* ...

* elide I generic when used with S: State

* Elide many, many generics, but at what cost?

* progress on push

* Constraint HasCorpus, HasSolutions at trait definition

* remove unused feature

* remove unstable usage since we constrained HasCorpus at definition

* compiled, but still no type inference for MaxMapFeedback

* cleanup inprocess

* resolve some std conflicts

* simplify map

* undo unnecessary cfg specification

* fix breaking test case for CI on no-std

* fix concolic build failures

* fix macos build

* fixes for windows build

* timeout fixes for windows build

* fix pybindings issues

* fixup qemu

* fix outstanding local build issues

* maybe fix windows inprocess

* doc fixes

* unbridled fury

* de-associate State from Feedback, replace with generic as AT inference is not sufficient to derive specialisation for MapFeedback

* merge update

* refactor + speed up fuzzer builds by sharing build work

* cleanup lingering compiler errors

* lol missed one

* revert QEMU-Nyx change, not sure how I did that

* move HasInput to inputs

* HasInput => KnowsInput

* update bounds to enforce via associated types

* disentangle observers with fuzzer

* revert --target; update some fuzzers to match new API

* resolve outstanding fuzzer build blockers (that I can run on my system)

* fixes for non-linux unixes

* fix for windows

* Knows => Uses, final fixes for windows

* <guttural screaming>

* fixes for concolic

* loosen bound for frida executor so windows builds correctly

* cleanup generics for eventmanager/eventprocessor to drop observers requirement

* improve inference over fuzz_one and friends

* update migration notes

* fixes for python bindings

* fixes for generic counts in event managers

* finish migration notes

* post-merge fix

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2022-10-24 03:22:26 +02:00
Mrmaxmeier
cedcee01c0
CI: Build fuzzers with shared cargo target dir (#845)
* build fuzzers with shared cargo target dir

* Make external build scripts aware of CARGO_TARGET_DIR

* fix libmozjpeg fuzzer with shared target dir

* fix cargo-make default value for CARGO_TARGET_DIR

* avoid ./ in cargo-make for windows compat

* CI: cargo-hack's --feature-powerset is too powerful

* fuzzer_concolic: support CARGO_TARGET_DIR

* ci: install z3 to avoid building from source

* ci: update actions

* ci: test nightly features with nightly rust

* test_all_fuzzers: try pruning more compilation artifacts

* ci: fix nightly feature check

* ci: apply rust-cache action after checkout (d'oh)

The rust-cache action populates the checkout directory, which is promply
deleted by the checkout action during checkout.. whoops!
2022-10-20 21:38:58 +02:00
Dominik Maier
e8b3d33bf4
Update dependencies, removed unused deps, CI fixes (#839)
* update clap, remove unused deps

* update grammartek

* update pyo3

* update pyo3

* undid clap update

* not changing nyx

* updated deps

* Update more deps, fixes

* not needed clippy

* fix windows

* try to enable deprecated pyproto for pyo3

* unused

* moving some things to clap4 after all

* initial move to clap 4

* fix clap

* more clap4, removed accidental file

* fixes, fmt

* fix

* all fix no play

* fix
2022-10-18 20:36:43 +02:00
Mrmaxmeier
f5cc354102
Token mutations: set MutationResult for CmpValues::Bytes (#838)
* token mutations: set MutationResult for CmpValues::Bytes

I haven't measured this and am not even sure if CmpValues::Bytes is
currently populated by any executor, but this seems like an oversight.

* replace dead zlib-1.2.12.tar.gz URL

See https://zlib.net/fossils/OBSOLETE
2022-10-14 13:03:57 +02:00
Andrea Fioraldi
089bc49d55
Bump to 0.8.2 and update versions script (#828) 2022-10-12 14:57:08 +02:00
Dongjia "toka" Zhang
6dc7cc2f59
bump (#799) 2022-09-25 09:41:01 +02:00
Dongjia Zhang
eac7307c5a
0.8.1 (#732) 2022-08-18 10:23:57 +02:00
Dongjia Zhang
ce5ac3968d
Fixing readme & docs (#730)
* fix

* fix

* add

* add

* fmt
2022-08-15 02:12:35 +02:00
syheliel
2504b6dae3
Add rustfmt.toml (#722)
* add `rustfmt.toml`

* format fix

Co-authored-by: syheliel <syheliel@gmail.com>
2022-08-12 02:28:32 +02:00
Dominik Maier
12052b5f1c
Remove num_cpus dependency (#717)
* Remove num_cpus dependency

* Fix build, remove more num_cpus
2022-08-03 07:49:32 +02:00
Dongjia Zhang
376e3adfcd
Bump Frida, Capstone versions (#715)
* bump

* fix

* fix

* revert

* fix

* fmt

* fix
2022-08-01 16:53:39 +02:00
Dominik Maier
5a8bdae26f
Update requirements (#714)
* Update requirements

* more updates

* nits

* more updates

* update nix for fuzzers

* use any regex 1 instead of 1.6
2022-07-29 09:54:16 +02:00
Dongjia Zhang
ffe8dbf6af
Bump to 0.8.0 (#696)
* upd

* more
2022-07-11 21:59:11 +02:00
Dongjia Zhang
bc6a032843
Windows CI for frida (#658)
* harness & makefile.toml

* Update build_and_test.yml

* Update build_and_test.yml

* Update build_and_test.yml

* rename

* no stdout

* Update harness_win.cpp

* Update harness_win.cpp

* Update Makefile.toml

* Update build_and_test.yml

* Update Makefile.toml

* Update Makefile.toml

* fix
2022-05-30 23:06:37 +02:00
Dominik Maier
bfe69aea09
Format C/Cpp code in ./scripts/fmt_all.sh (#653)
* format all (clang format 13)

* added clang-format check

* re-add missing newline

* cargo doc was missing

* more brackets

* fixed fmt workflow

* clang format

* shellcheck

* install clang-format-13

* update ubuntu for maximum clang-formattability

* yml whitespaces

* fmt

* shellcheck only for .sh

* oops path

* ignored shellcheck warning
2022-05-29 03:23:02 +02:00