228 Commits

Author SHA1 Message Date
Dongjia "toka" Zhang
c3930b39fc
Fix CI (#2312) 2024-06-14 14:16:31 +02:00
Dongjia "toka" Zhang
c415b4d5f6
0.13.0 (#2253)
* 0.13

* z3

* capstone

* fixer

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-06-13 23:35:35 +02:00
Dongjia "toka" Zhang
0ed295842b
Use filename for MmapShMemProvider (#2303)
* fixer

* comment

* a

* bb

* apple
2024-06-11 20:15:11 +02:00
Dongjia "toka" Zhang
f4699ba385
Make adaptive serialization into default (#2296)
* no adaptive

* add another api

* allow unused
2024-06-09 00:10:30 +02:00
Dongjia "toka" Zhang
454176427b
Windows clippy (#2295)
* add

* real one

* fuck

* abc

* def

* ghi

* jkl

* fix

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-06-08 20:32:40 +02:00
Dongjia "toka" Zhang
1e2fac6f53
abort() when the weakly defined LLVMFuzzerTestOneInput is linked(#2289) 2024-06-07 23:54:19 +02:00
Romain Malmain
af12b4ba24
LLmp hooks (#2280)
* llmp hooks

* separate llmp broker from inner state to allow doing more stuff
2024-06-06 16:07:55 +02:00
Dongjia "toka" Zhang
6373a1e1b3
Use Self::State as much as possible and remove unnecessary type bounds (#2263)
* use Self::State as much as possible

* makeing progress, little by little

* more

* add

* more and more

* more

* more

* mre

* fix

* a

* pp

* fix

* fix

* more

* version

* fix cargo fuzz
2024-06-05 15:16:43 +02:00
Aarnav
bce0f08294
Introduce feedbacks hit tracking for testcases (#2248)
* introduce feedbacks hit tracking for testcases

* make Testcase::hit_feedbacks into Cow<&str> instead of String
rename get_hit_feedbacks to append_hit_feedbacks
update documentation

* simplify ConstFeedback

* rename Feedback::last_result to prev_result

* impl TODO prev_result for NewHashFeedback, ListFeedback, TransferredFeedback, NautilusFeedback

* rename prev_result to last_result

* add docs

* introduce Objectives hit tracking

* update docs

* update Cargo.toml docs

* update docs

* track Feedbacks & Objectives hit in Fuzzer::add_input

* fmt

* clippy

* fix type error in OomFeedback::last_result

* impl last_result for AsanErrorsFeedback

* add track_hit_feedbacks as a feature to libafl_libfuzzer_runtime

* fix clippy

* change return type of Feedback::last_result to a Result

* remove expect in NewHashFeedback::is_interesting

* move Error::premature_last_result to libafl from libafl_bolts
2024-05-28 11:43:05 +02:00
Dongjia "toka" Zhang
e6eb6c48d2
LibAFL Dynamic Analysis (#2208)
* aa

* push

* push

* push

* claude ai

* push

* add

* add stuff

* upd

* rdm

* fix

* ci

* fix

* fix

* fixing

* feature

* revert

* no submodules

* ci
2024-05-24 14:43:27 +02:00
Dominik Maier
1fafaf6454
Make every Builder ::builder(), so BobTheBuilder::new() becomes BobThe::builder() (#2242)
* Make every builder ::builder()

* Fix no_std

* More

* Fix clippy, stuff

* More fun

* Make NopShMem do something

* Alloc

* more fmt

* Remove UB in tinyinst executor builder

* Make builder order not matter for tinyinst

* More better

* fix

* docs

* fmt

* more fmt

* clippy

* fix fixes

* tiny thing

* more betterg

* more more

* more builder

* more builder

* more nyx

* undo breaking clippy

* clip
2024-05-23 18:56:39 +02:00
Dominik Maier
684b31279e
Add BytesSubInput to mutate sub-parts of a bytes-backed input (#2220)
* Add BytesSubMutator that allows us to mutate sub-parts of a bytes-backed input

* no_std

* fix string mutator

* make build

* Fix clippy on macOS

* Docs

* More docs

* Better docs

* --amend

* Renamed bsi to sub_input. Too much BSI

* More more

* balance backticks

* Make splicing usable with sub_input (not that it makes sense)

* More annotations

* more input annotations?

* Implement HasMutatorBytes for &mut Vec

* clippy

* Use a wrapper type instead

* Add wrapper type for Vec as well

* Remove the duplicate BytesInput... lol
2024-05-22 01:50:07 +02:00
Dongjia "toka" Zhang
19ef29ed60
New forkserver (#2213)
* step 1

* done

* cmplog?

* targets

* check if working and add env_logger

* typo
2024-05-21 15:20:28 +02:00
s1341
19087f3dab
Windows frida support (#1607)
* WIP: windows frida

* frida-windows: fix hooks not present on windows

* windows: allow building using cargo xwin

* frida-windows: fmrt

* frida-windows: cleanup and allow asan/drcov on windows

* frida-windows: fmt

* frida-windows: fix clippy

* frida-windows: handle unknown exceptions gracefully

* frida-windows: rework shadow mapping algo

* frida-windows: add hook functions

* frida-windows: hook functions; fix stack register

* minibsod: enable for windows

* check_shadow: fix edge casees

* asan_rt: rework and add hooks for windows

* inprocess: add minibsod on windows

* Fix warnings

* minibsod: disable test on windows

* WIP: HookRuntime

* Cleanup after merge

* Bump frida-gum version

* Fix conflict marker; update frida

* Make winsafe windows-specific

* Fmt

* Format

* Better detection of clang++ (using cc)

* Make AsanErrors crate public so we can use it in tests

* Add helper to get immediate of operand

* Use HookRuntime to hook asan functions

Tests now passing

* fmt

* Implement recurisve jmp resolve

* Fix reversed logic

* windows_hooks: Don't die if functions are already replaced

* Allow utils to work on windows

* Enable allocator hooking on windows

* Warnings; add trace to free

* Make ASAN tests run windows (with cargo xwin compilation)

* Fmt

* clang-format

* clang-format

* Add more tests

* Fix partial range access bug in unpoisoning/shadow_check

* Merge main

* Fix check_shadow and implement unit tests

* Fix hooking and PC retrieval

* WIP: Working gdiplus fuzzing with frida-ASAN, no false positives

* LibAFL Frida asan_rt and hook_rt fixes for frida_windows (#2095)

* Introduce aarch64

* MacOS fix - MemoryAreas is broken on MacOS and just loops

* Introduce working aarch64 ASAN check

* Implement large blob

* Fix hook_rt for arm64

* Fix poison/unpoison

* Fix shadow check

* Update x86-64

* Fix aarch64 unused import

* Remove extraneous println statement

* merge main

* Fixes

* alloc: add tests, pass the tests

* HookRuntime before AsanRuntime, and don't Asan if Hooked

* hook_rt: Fixes

* Frida windows check shadow fix (#2159)

* Fix check_shadow and add additional tests

* add some additional documentation

* Revert to Interceptor based hooks

* fixes

* format

* Get rid of hook_rt; fixes

* clang-format

* clang-format

* Fix with_threshold

* fixes

* fix build.rs

* fmt

* Fix offset to RDI on stack

* Fix clippy

* Fix build.rs

* clippy

* hook MapViewOfFile

* fmt

* fix

* clippy

* clippy

* Missing brace

* fix

* Clippy

* fomrrat

* fix i64 cast

* clippy exclude

* too many lines

* Undo merge fails

* fmt

* move debug print

* Fix some frida things

* Remove unused frida_to_cs fn for aarch64

* name

* Don't touch libafl_qemu

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Sharad Khanna <sharad@mineo333.dev>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2024-05-14 10:45:56 +02:00
Addison Crump
614b149cae
Fix cfg uses across codebase (#2154)
* start fixup of cfgs

* whoops

* remaining issues

* maybe fixup macos

* apparently that is ios code?

* fix no_std

* Re-enable tcp_compression

* fix tcp broker

* remove tcp_debug flag

* fmt

* clippy

* less unwrap while we're at it

* doc

* add back pub star uses for libafl_qemu

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-05-08 14:31:10 +02:00
Dominik Maier
c97c8cf83e
Rename Handler -> Handled, various other cleanups after #2142 (#2152)
* Handle name cleanup after #2142

* Handleable -> CanBeHandled

* CanBeHandled -> Handled

* clippy

* forgot one

* Handled more
2024-05-07 19:23:13 +02:00
Dongjia "toka" Zhang
b0ac25a7b5
Rename Reference to Handle (#2142)
* rename

* rename

* fmt

* fix

* fmt
2024-05-04 18:16:28 +02:00
Dongjia "toka" Zhang
50f14d44fa
Restrict more on edges map API (#2117)
* fix

* fix

* feature guard

* use

* cannnot find

* fmt

* more

* fix

* fix

* more

* fucking fmt just shut

* rename and change stuff

* fix

* fix

* fixxxxxxxxxx
2024-04-29 16:28:40 +02:00
Langston Barrett
b0248461e2
Allow for AsIter(Mut)/AsSlice(Mut) to be implemented in safe Rust (#2120)
* Generalize `AsIter` to allow iterating over contents of a `RefCell`

Towards `MapObserver`s in safe Rust.

* Helpers for `RefCellValueObserver`

* MapObserver: Return owned Self::Entry from .get()

`Self::Entry` is `Copy`, so there's not much value in returning a
reference from `get()`. Futhermore, returning a reference limits the
possible implementations of `MapObserver`, because it forces the
borrow/reset to outlive the body of the method.

* MapObserver: Replace `.get_mut()` with `.set(idx, val)`

Like the previous commit, this is intended to expand the possible
implementations of `MapObserver` to types with interior mutability,
which can't necessarily loan out their content.

* Make `RefCellValueObserver` into a safe `MapObserver`

* as iter mut

* as slice (mut): allow for non-& refs

* nostd

* CI round 1

* cleanup + AsSlice defs for RefCellValueObserver

* clippy fixes

* avoid unnecessary imports

* whoops, too aggressive

* use deref instead of as slice

* whoops

* fix as slice conditional importing in stable

---------

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2024-04-27 18:36:56 +02:00
Addison Crump
28c43b332f
Match by Ref fix (#2105)
* match by ref fix

* impl Named for Reference

* rename

* magic indexing

* whoops

* docs, clippy

* some additional CI complaints

* other libafl_qemu fixes

* missed an alloc feature

* a smattering of fixes

* use from not direct construction

* tinyinst fix

* horrible double-mutability things

* fixup nyx

* from not new

* forkserver_simple fixes

* dogfood: forkserver

* mmmm yummy dogfood

* round one CI fixes

* clippy appeasement

* deref generic impl to simplify usage

* adaptive serialization (ouch)

* remaining clippy items

* I am tired

* new not with

* fixup: aflpp tracing was not actually constructable

* fix tmin

* reduce complexity of map feedback now that we do not need to constrain

* frida fixes

* fix concolic

* type_ref => reference
2024-04-27 18:08:09 +02:00
Dongjia "toka" Zhang
176659821a
Add TypeRefCreator trait (#2065)
* poc

* ai suggestion

* rename this

* aaaa

* fmt

* simplify

* delete blob

* ignore

* fixup?

* some progress on cow-ification

* some more

* clippy fixes, finalise tests

* whoops, missed a spot

* no std compat

* api change: Named now requires alloc feature

* doc fix

* missed a spot

* additional fixes

* libfuzzer fixes

* fix tutorial

* fix

* add

* aa

* fix tutorial

* fix

* Rename

* fix

* aa

* fmt

* aa

---------

Co-authored-by: Addison Crump <addison.crump@cispa.de>
2024-04-23 19:03:30 +02:00
Dongjia "toka" Zhang
5ff709f241
Symcc Update (#2047)
* symcc upd

* upda

* fix

* fmt

* mm

* psu

* psh

* fix

* fix

* fix

* tmate

* can't make it work

* i really hate you llvm

* real name

* mm
2024-04-23 13:34:46 +02:00
Romain Malmain
fef6c8d1b1
Run qemu_systemmode tests with self-hosted runners (#2018)
* run qemu fuzzers (qemu_systemmode only for now) in self-hosted runners

* Remove qemu-related fuzzers to general fuzzers

* fix

* Install dependencies before anything else

* Do not use sudo

* Install sudo

* Revert "Install dependencies before anything else"

This reverts commit 107addad5d9f68dec5a9af50831112cd72c28f4d.

* added qemu specific prerequisites

* add -y flag

* Format with nightly

* Do not use nightly only.
Install fmt and clippy for stable as well.

* Install qemu-img for qemu

* fix qemu-img install

* apt update

* Changed timeout.

* Fix qemu_systemmode test

* fmt

* clippy + decorrelate build and run for qemu_systemmode.

* fix fuzzer

* clippy

* add sqlite3-dev to package prerequisites.

* add arm-none-eabi-gcc

* fix profile dir

* fix condition.

* Run less QEMU stuff faster

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-04-22 18:17:22 +02:00
Dongjia "toka" Zhang
e8cc1ba93d
Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use" (#2082)
* Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2…"

This reverts commit c68b3816fb680b635f99c337f78185e699864705.

* all

* more
2024-04-19 19:03:58 +02:00
edwin1729
c238b69498
Implement Hash for MapObserver (#1989)
* MapObserver implements Hash

* Rename the hash utility function (in MapObserver) to hash_easy

* Use hash_slice as a helper function to impl Hash trait

* define_python_map_observer macro implements Hash trait

* Also rename hash_easy to hash_simple

* Rename hash_slice to hash_helper

* hash_helper is used to define the implementation of hash function/trait

* Factor out the Hash trait and function for runtime library structs (#1977)

* Simplify hash_simple (of trait MapObserver) (#1977)

 * Use hash_one function to make hash_simple a one-liner

* remove hash_helper

---------

Co-authored-by: Edwin Fernando <ef322@ic.ac.uk>
Co-authored-by: Addison Crump <addison.crump@cispa.de>
2024-04-19 15:06:14 +02:00
Dongjia "toka" Zhang
c68b3816fb
Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2079)
* fix

* aa

* push

* fix

* fmt
2024-04-19 13:24:25 +02:00
Romain Malmain
84a432dab2
fix import (#2076) 2024-04-18 15:43:21 +02:00
Romain Malmain
39c32d55e7
Fix main not passing CI tests (#2075)
* Fix sancov import

* fmt

* fmt
2024-04-18 13:33:18 +02:00
mkravchik
886519b10c
Pr/fixing forkserver libafl cc (#2066)
* Fixing forserver_libafl_cc

* Adding tests and showing user stats

* Restoring the map truncation

* Fmt

* small fix

* fix

* fix

* fix

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2024-04-17 18:28:34 +02:00
Dongjia "toka" Zhang
c50af44099
Split edges_map_size and edges_map_size_alloc (#2012)
* two sizes

* rename

* build.rs

* rename

* capital

* FMT

* macro

* fuck

* fff
2024-04-17 17:58:08 +02:00
Dongjia "toka" Zhang
8bce605503
DDFuzz (#2056)
* push

* add

* book

* ddg

* works
2024-04-16 16:51:28 +02:00
Dongjia "toka" Zhang
cbb323f26c
LibAFL 0.12 (#1823) 2024-04-12 18:42:14 +02:00
Addison Crump
3d702f403b
Implement CanTrack tracking enforcement through rust types (#1886)
* sample implementation of tracking enforcement (incomplete)

* helpful compiler output

* make it look like a real compiler output

* ensure that the macro may be used outside of libafl

* separate index/novelty tracking funcs

* default const generic values so that we don't need to change this everywhere

* fix tests

* rollback unnecessary specification of stdmapobserver

* register metadata in doc tests

* doc fixes

* doc cleanup

* doc cleanup 2

* reduce implementor overhead to zero

* renaming/docs fixes

* asref isn't reflexive??

* generalization stage updates

* add better documentation about require_{indices,novelties}_tracking

* remaining generic updates

* round one CI pass (knowingly introduces breaking changes)

* typo

* round 2 clippy

* rollback: libafl_frida changes

* fmt

* moar porting

* fix remaining fuzzers

* fix windows build, maybe

* fixup libafl_libfuzzer

* fmt nighlty all the things

* attempt to fix some broken additions

* fix fmt

* oops

* fix new invocation

* minimizer scheduler fixes

* fix accounting

* rename

* fix

* Fix build

* Sort generics

* Move more generics into the right place

* Rename A -> C

* Fix test

* Fix test some more

* Fix doc some more

* critical formatting

* More A->C

* CanTrack harder

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-04-12 18:33:52 +02:00
Aarnav
47c41c2925
Introduce disabled testcases for splicing (#1932)
* introduce disabled field to Testcase

* separate executor's processing of execution (adding to corpus/solution/discarding) and execution of input

* introduce add_disabled_input function

* enable splicing mutators to fetch disabled inputs

* reset modified example

* clean up

* update docs

* update docs for count_with_disabled

* fix random_corpus_id for splicing mutator not considering disabled entries

* fmt

* update docs

* clippy

* fix corpus_btreemap not working

* fix clippy warnings

* fix python bindings

* typo in count_with_disabled implementations

* fix certain splicing mutators not considering disabled inputs

* rename count_with_disabled to count_all

* introduce count_disabled function

* update docs for count_all, count_disabled and count

* * introduce get_from_all and nth_from_all for corpus implementations so get() and nth() do not silently fetch disabled entries.
* remove boolean flag from random_corpus_id which allowed inclusion of disabled ids and make it into a new function random_corpus_id_with_disabled
* update docs

* remove boolean is_disabled flag from corpus::insert and make it into a separate function insert_disabled

* rename do_insert to _insert

* make get_from_all inline for cached and inmemory

* add missing functions implementation for PythonCorpus
prevent writing feedback when adding disabled testcases

* fix nth_from_all overfetching enabled corpus entries

* fix clippy & rename execute_with_res to execute_no_process

* refactor _insert for corpus_btreemap

* make LibfuzzerCorpus and ArtifactCorpus to accomodate disabled entries

* fix typo

* fix missing docs for map field

* fix clippy

* test

* (hopefully) fix CachedOnDiskCorpus using incorrect corpus when caching testcase

* fix typo in inmemory_ondisk leading to fetching disabled entry from enabled corpus

---------

Co-authored-by: aarnav <aarnav@srlabs.de>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-04-10 01:03:00 +02:00
Dongjia "toka" Zhang
e8fe5bb614
Move HasMetadata/HasNamedMetadata trait to libafl_bolts (#2016)
* move around

* fmt

* rev

* fmt

* doc job

* name

* move to libafl

* separate even more

* miri
2024-04-08 19:24:56 +02:00
Dominik Maier
ff938261df
Fix Clippy on MacOS for qemu, accept some clippy::missing_transmute_annotations suggestions (#2009)
* Fix Clippy/CI

* Ignore transmute annotations in pybind

* fix

* more clippy

* more clippy

* fix fixes
2024-04-08 16:24:10 +02:00
Dongjia "toka" Zhang
527b892c1d
Separate fuzzer test in CI (#2010)
* ci

* ci

* ci

* aa

* aa

* a

* a

* a

* fix

* no fail fast

* system

* what is this lint...

* a

* clp

* disk

* ok no fastfail

* no qemu for now, reenable monday
2024-04-06 21:01:57 +02:00
Dongjia "toka" Zhang
17aae94efc
Move some C functions to Rust from sancov_cmp.c (#2003)
* some

* a

* clp

* aa

* just new line

* fix

* fxi

* fix
2024-04-06 01:03:01 +02:00
Dongjia "toka" Zhang
91778961da
Inline cmplog internal functions code (#1972)
* fmt

* aa

* sattic

* win

* frida

* expose API

* fmt

* frida
2024-03-26 17:42:03 +01:00
Dominik Maier
04d87ccc89
Remove (almost) unused stage_idx (#1929)
* Remove (almost) unused stage_idx

* Fix text-based test

* fixed critical whitespace

* clippy

* more fmt

* fix push stage
2024-03-14 13:14:57 +01:00
Dongjia "toka" Zhang
afa2965f3c
Remove some arguments from pre_exec/post_exec in ExecutorHook (#1933)
* refactor

* no std, fmt

* win mac

* state

* frida

* ctx

* trait bound

* clip
2024-03-13 18:49:09 +01:00
Dominik Maier
dd410c590a
Allow dyn in StagesTuple, add Current Testcase API, Untraitify Progress (#1915)
* Move into_vec to extra trait

* fix no_std

* First step towards stages tuples as vecs

* Allow dyn in StagesTuple, add Current Testcase API, un-traitify Progress, cleanups

* Move from generics to impl keyword, more replacements with better API

* rename fn

* Fix additional stages, more cleanup, rename progress to retries

* Fix more fixes

* Fixes

* Rename ProgressHelper -> RestartHelper

* Fix sugar, python, add perform_restartable

* fixes

* remove prelude bs

* rename to restart_progress_should_run

* more cleanup, remove tests I don't understand (sorry)

* fix docs

* more fix

* fix miri

* unsafe safety annotations

* more comments

* last docs

* Mixed_attributes only allowed for bindgen
2024-03-12 00:58:07 +01:00
Dongjia "toka" Zhang
9b780cc0a7
Fix UB in anymap.rs and other minor warnings. (#1926)
* fix

* clippy

* fix
2024-03-11 18:15:56 +01:00
Romain Malmain
e745401a39
Update QEMU version. (#1922)
* update QEMU version.

* clippy
2024-03-08 13:15:00 +01:00
Dongjia "toka" Zhang
9a2e7b0e64
Fix redundant xor in Ngram calculation (#1901)
* fix double xor

* tmp

* clp

* no notnightly

* fix

* plural
2024-03-07 14:49:01 +01:00
Rayhan Faizel
44b0b0ad82
libafl_targets/src/cmplog.c: Fix warning for deprecated syscall() in macOS (#1911) 2024-03-05 19:04:20 +01:00
Romain Malmain
d96a1426d5
Fix lint errors (#1909)
* fix lints.

* more lint fix.

* even more lint fixes.

* always more lint fixes.

* lint fix.

* allow unused qualifications for crate when it could be confusing.

* Still lint fixes.

* Lint fixes on generated code.

* Some lint fixes.
2024-03-05 14:16:26 +01:00
Dongjia "toka" Zhang
1a41e65a5b
Sancov based ngram & ctx implementation (#1864)
* ngram

* ctx

* push stuff

* passing cargo check

* save stuf

* add

* no default

* fuzzbench ready

* formatt

* aaaaaaaa

* adaptive map

* add all

* chg

* fix

* deleting stuff

* fmt

* clang18

* fuck

* save space

* fix?

* feature name

* fucking nightly only

* harness

* fix libafl_cc

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-03-01 13:59:13 +01:00
Dongjia "toka" Zhang
b97598d239
Windows fix (#1876)
* win fix

* just some ci

* test

* revert

* aaaaa

* bbbbbb
2024-02-22 14:56:30 +01:00
Sameer Puri
50892ddc75
Finalize macOS support for libafl_libfuzzer (#1843)
* libafl_targets: on macOS, do not provide a default implementation for weak functions

* libafl_libfuzzer: update README to talk about macOS specifics

* libafl_targets: allow __sanitizer_cov_pcs_init to be called more than once

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-02-15 22:45:15 +01:00