2845 Commits

Author SHA1 Message Date
Sharad Khanna
8df9e68677
Move test_harness from source directory to OUT_DIR (#2694)
* remove test_harness from source directory

* fmt
2024-11-14 12:47:45 -03:00
Mrmaxmeier
8f320d4314
StdMOptMutator:🆕 remove unused type parameter (#2695)
`I` is unused in `::new` and thus requires callers to explicitly specify
any type as it can't be determined by type inference.

Clippy's `extra_unused_type_parameters` should pick this up, but is
tuned a bit too conservative in order to avoid false positives AFAICT.
2024-11-14 12:46:28 -03:00
Andrea Fioraldi
b9cfcf0771
Fix versions in libafl and libafl_intelpt for crates.io (#2693)
* Fix versions in libafl and libafl_intelpt for crates

* Add libafl_intelpt to publish
2024-11-14 00:06:36 +01:00
Andrea Fioraldi
d339a58f7c
Bump to 0.14.0 (#2692) 2024-11-13 23:10:35 +01:00
jejuisland87654
8d81e3480e
Fix NautilusContext::from_file for python files (#2690) 2024-11-13 16:45:18 -03:00
Marco C.
b7889a5996
Move bitfields to bitbybit (#2688)
* move to bitbybit

* Restore bitbybit dependent code

* Clippy
2024-11-13 16:28:25 -03:00
Aarnav
d334860148
Feature: libafl-fuzzfuzzbench (#2689)
* fuzzbench

* clippy

* fmt

* fix unicorn CI?
2024-11-13 13:32:21 +01:00
Dominik Maier
7938acc4ce
Actually make ConstMapObserver work, introduce nonnull_raw_mut macro (#2687)
* Actually make ConstMapObserver work

* fixes

* does that work?

* mas
2024-11-13 00:35:46 -03:00
Dongjia "toka" Zhang
d5db2c0e3f
LibAFL: Remove tui_monitor from default features (#2685)
* No Usermode default

* no tui

* gg

* try fix CI

* fmt

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-11-12 22:55:02 -03:00
Aarnav
87f5f21e01
libafl-fuzz: introduce nyx_mode (#2503)
* add nyx_mode

* fix frida ci?

* damn clippy

* clippy
2024-11-12 22:51:07 -03:00
Marco C.
f7f8dff6cd
Add Intel PT tracing support (#2471)
* WIP: IntelPT qemu systemmode

* use perf-event-open-sys instead of bindgen

* intelPT Add enable and disable tracing, add test

* Use static_assertions crate

* Fix volatiles, finish test

* Add Intel PT availability check

* Use LibAFL errors in Result

* Improve filtering

* Add KVM pt_mode check

* move static_assertions use

* Check for perf_event_open support

* Add (empty) IntelPT module

* Add IntelPTModule POC

* partial ideas to implement intel pt

* forgot smth

* trace decoding draft

* add libipt decoder

* use cpuid instead of reading /proc/cpuinfo

* investigating nondeterministic behaviour

* intel_pt module add thread creation hook

* Fully identify deps versions

Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates

* Move mem image to module, output to file for debug

* fixup! Use static_assertions crate

* Exclude host kernel from traces

* Bump libipt-rs

* Callback to get memory as an alterantive to image

* WIP Add bootloader fuzzer example

* Split availability check: add availability_with_qemu

* Move IntelPT to observer

* Improve test docs

* Clippy happy now

* Taplo happy now

* Add IntelPTObserver boilerplate

* Hook instead of Observer

* Clippy & Taplo

* Add psb_freq setting

* Extremely bad and dirty babyfuzzer stealing

* Use thread local cell instead of mutex

* Try a trace diff based naive feedback

* fix perf aux buffer wrap handling

* Use f64 for feedback score

* Fix clippy for cargo test

* Add config format tests

* WIP intelpt babyfuzzer with fork

* Fix not wrapped tail offset in split buffer

* Baby PT with raw traces diff working

* Cache nr_filters

* Use Lazy_lock for perf_type

* Add baby_fuzzer_intel_pt

* restore baby fuzzer

* baby_fuzzer with block decoder

* instruction decoder instead of block

* Fix after upstream merge

* OwnedRefMut instead of Cow

* Read mem directly instead of going through files

* Fix cache lifetime and tail update

* clippy

* Taplo

* Compile caps only on linux

* clippy

* Fail compilation on unsupported OSes

* Add baby_fuzzer_intel_pt to CI

* Cleanup

* Move intel pt + linux check

* fix baby pt

* rollback forkexecutor

* Remove unused dep

* Cleanup

* Lints

* Compute an edge id instead of using only block ip

* Binary only intelPT POC

* put linux specific code behind target_os=linux

* Clippy & Taplo

* fix CI

* Disable relocation

* No unwrap in decode

* No expect in decode

* Better logging, smaller aux buffer

* add IntelPTBuilder

* some lints

* Add exclude_hv config

* Per CPU tracing and inheritance

* Parametrize buffer size

* Try not to break commandExecutor API pt.1

* Try not to break commandExecutor API pt.2

* Try not to break commandExecutor API pt.3

* fix baby PT

* Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620)

* support (unsafe) on_crash / on_timeout callbacks for modules

* use libc types in bindgen

* Move common code to bolts

* Cleanup

* Revert changes to backtrace_baby_fuzzers/command_executor

* Move intel_pt in one file

* Use workspace deps

* add nr_addr_filter fallback

* Cleaning

* Improve decode

* Clippy

* Improve errors and docs

* Impl from<PtError> for libafl::Error

* Merge hooks

* Docs

* Clean command executor

* fix baby PT

* fix baby PT warnings

* decoder fills the map with no vec alloc

* WIP command executor intel PT

* filter_map() instead of filter().map()

* fix docs

* fix windows?

* Baby lints

* Small cleanings

* Use personality to disable ASLR at runtime

* Fix nix dep

* Use prc-maps in babyfuzzer

* working ET_DYN elf

* Cleanup Cargo.toml

* Clean command executor

* introduce PtraceCommandConfigurator

* Fix clippy & taplo

* input via stdin

* libipt as workspace dep

* Check kernel version

* support Arg input location

* Reorder stuff

* File input

* timeout support for PtraceExec

* Lints

* Move out method not needing self form IntelPT

* unimplemented

* Lints

* Move intel_pt_baby_fuzzer

* Move intel_pt_command_executor

* Document the need for smp_rmb

* Better comment

* Readme and Makefile.toml instead of build.rs

* Move out from libafl_bolts to libafl_intelpt

* Fix hooks

* (Almost) fix intel_pt command exec

* fix intel_pt command exec debug

* Fix baby_fuzzer

* &raw over addr_of!

* cfg(target_os = "linux")

* bolts Cargo.toml leftover

* minimum wage README.md

* extract join_split_trace from decode

* extract decode_block from decode

* add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0

* More generic hook

* fix windows

* Update CI, fmt

* No bitbybit

* Fix docker?

* Fix Apple silicon?

* Use old libipt from crates.io

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-11-12 22:34:46 -03:00
Dongjia "toka" Zhang
5eff9c03d3
Fix build_all_fuzzers.sh for local runs (#2686) 2024-11-12 22:19:26 -03:00
Dominik Maier
7fada7d985
Add DrCovReader to read DrCov files and DrCov dumper and merge utils (#2680)
* Add DrCov Reader

* Removed libafl_jumper deps

* Fix DrCovWriter, add dump_drcov_addrs

* Taplo

* Move frida from usize to u64

* DrCov usize=>u64

* Better error print

* More u64

* ?

* debug

* clippy

* clippy

* Add Merge option to DrCovReader

* Add drcov_merge tool

* Move folder around

* DrCov

* More assert

* fmt

* Move around

* Fix print

* Add option to read multiple files/full folders
2024-11-12 22:18:57 -03:00
Dominik Maier
0ef0684e43
LibAFL_QEMU: Don't return a generic Address from Register reads (#2681)
* LibAFL_QEMU: Make ReadReg always return GuestReg type

* Don't return a generic address

* fix fuzzers

* fix mips
2024-11-12 22:14:35 -03:00
Marco C.
f3aa88b400
Use version.workspace (#2682) 2024-11-12 15:45:35 +01:00
Valentin Huber
ae9ab80ad9
Fixing version pinning in CI (#2679) 2024-11-12 00:49:58 +01:00
jejuisland87654
4269be1c08
fix error '#' is not followed by a macro parameter (#2678) 2024-11-11 19:43:27 +01:00
Subhojeet Mukherjee, PhD
e25094eb4f
Lower capped RAND generators (#2671)
* Lower capped rand generators

* Updated all references to RAND generators

* Formatting updates

* New RAND bytes generator constructor

* Revert "Updated all references to RAND generators"

This reverts commit 9daad894b25ec3867daf93c4fe67c03abec1d8c6.

* Revert "Formatting updates"

This reverts commit ff2a61a366c48b3f313878f62409e51b1e1ed663.

* cargo nightly format

* Added must_use to with_min_size
2024-11-11 19:36:55 +01:00
Valentin Huber
e32b3eae93
Introducing Launcher::overcommit, improving CI formatting (#2670)
* introducing Launcher::overcommit

* removing unnecessary cfg restrictions and clippy allows

* improving warning for wrong clang-format version

* installing black in the format CI

* Enforcing python formatting in CI

* extending formatting using black on all python files

* printing diff on black failure

* preferring python's black over system black

* moving to LLVM 19 for formatting
2024-11-09 19:13:51 +01:00
Dominik Maier
8617fa6603
Introduce workspace (again) (#2673)
* Trying to redo workspace deps again after #2672

* unused

* clippy
2024-11-09 19:11:01 +01:00
Dominik Maier
21f8b1d147
Replace addr_of with &raw across the codebase (#2669)
* Replace addr_of with &raw across the codebase

* fix fixes

* more fix

* undo clang fmt?

* oops

* fix?

* allocator fix

* more fix

* more more

* more docs

* more fix

* mas mas mas

* hm

* more

* fix Frida

* needed

* more error

* qemu
2024-11-08 17:19:55 +01:00
Aarnav
d1c746a0a2
Set rlimit to inifinity for core dumps if AFL_DEBUG=1 (#2643)
* forkserver: set rlimit to inifinity for core dumps if AFL_DEBUG=1

* move coredump rlimit to a separate func

* update docs
2024-11-06 14:11:58 +01:00
bernhl
cfe124043e
Add memfd shmem backend (#2647) 2024-11-06 11:37:43 +01:00
Dominik Maier
36a24ab418
Custom Executor Example (#2570)
* [WIP] Custom Executor Example

* readme

* src/main.rs

* Finish

* fix warnings

* reame

* CI
2024-11-05 16:49:07 +01:00
Dominik Maier
b5c9bffe50
Core::errors::Error is stable now (#2664) 2024-11-05 16:48:31 +01:00
Dhanvith Nayak
8de9dcaff7
Clean up clippy warnings in fuzzers/binary_only/* (#2662)
* clean clippy warnings from fuzzers/binary_only/*

* handle unused Results in fuzzers/binary_only/*

* format fuzzers/binary_only/qemu_cmin

* use unchecked memory write in qemu fuzzer examples

* create file_null in fuzzbench_fork_qemu
2024-11-05 15:22:14 +01:00
dependabot[bot]
4581c50023
Update hashbrown requirement from 0.14.5 to 0.15.1 (#2660)
---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-05 14:46:40 +01:00
Valentin Huber
4431f069e5
Fix crossover mutators for empty multipart inputs (#2663) 2024-11-05 14:45:56 +01:00
Dongjia "toka" Zhang
87e4a845e0
Reenable libfuzzer_stb_image_sugar test (#2659) 2024-11-04 22:46:12 +01:00
Dongjia "toka" Zhang
e0c5304e32
Don't use workspace dependencies (#2652)
* rev

* postcard

* serde

* fmt

* a

* dependabot

* postcard again

* hashbrown

* upd

* add

* serde?
2024-11-04 17:47:40 +01:00
jejuisland87654
1e0b5581e5
better definition for PROFILE_DIR in all Makefile.toml files (#2658) 2024-11-04 17:09:12 +01:00
dependabot[bot]
cbe89b096d
Update ratatui requirement from 0.28.1 to 0.29.0 (#2657)
Updates the requirements on [ratatui](https://github.com/ratatui/ratatui) to permit the latest version.
- [Release notes](https://github.com/ratatui/ratatui/releases)
- [Changelog](https://github.com/ratatui/ratatui/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ratatui/ratatui/compare/v0.28.1...v0.29.0)

---
updated-dependencies:
- dependency-name: ratatui
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 17:08:04 +01:00
dependabot[bot]
51393242d8
Update dynasmrt requirement from 2.0.0 to 3.0.1 (#2653)
* Update dynasmrt requirement from 2.0.0 to 3.0.1

Updates the requirements on [dynasmrt](https://github.com/CensoredUsername/dynasm-rs) to permit the latest version.
- [Changelog](https://github.com/CensoredUsername/dynasm-rs/blob/master/doc/releasenotes.md)
- [Commits](https://github.com/CensoredUsername/dynasm-rs/compare/v2.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: dynasmrt
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix for update

* fix

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toka <tokazerkje@outlook.com>
2024-11-04 16:32:20 +01:00
dependabot[bot]
d7ce89d78d
Update goblin requirement from 0.8.2 to 0.9.2 (#2655)
Updates the requirements on [goblin](https://github.com/m4b/goblin) to permit the latest version.
- [Changelog](https://github.com/m4b/goblin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/m4b/goblin/commits)

---
updated-dependencies:
- dependency-name: goblin
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 16:23:27 +01:00
Romain Malmain
49ea0b03a6
Compile-time edge module compilation check, native support for ConstMapObserver (#2592)
* compile-time edge module compilation trick

* clippy

* possible since rust 1.79

* split edge module in submodules

* Update frida to 0.14.0 (#2596)

* update frida crate to the latest version

* adapt libafl_frida to the latest version of frida

* tracers and generators private modules

* do not use star export.

* same for drcov

* forgot a file...

* first draft of generic-based edge module for ConstantLengthMapObserver.

* integration of OwnedSizedSlice.

replaced OwnedSlice in ConstMapObserver by the new OwnedSizedSlice.

* fix serde stuff

* no std

* import

* fixed qemu_cmin with new constant map abstraction.

* fix const map

* fix clippy from another pr...

* fix non-null usage

* fix ci?

* new feature stuff

* fixes

* minor fixes

* fmt

* non null

* im stupid

* fmt

* fix fuzzer

* fix fuzzers

* sized slice

* fuzzer fixes

* ptr::NonNull -> NonNull

* shorter trait length

* fmt
2024-11-04 14:34:52 +01:00
Marco C.
56a5463ae4
Use a proper pre-commit hook for taplo fmt (#2650)
* Add taplo to pre-commit

* Use a proper pre-commit hook for taplo fmt
2024-11-04 14:17:02 +01:00
Yufei Li
97a8e4c294
Add RISCV support in libafl_qemu.h (#2380)
* Add riscv support in libafl qemu header

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-11-04 11:10:25 +01:00
Romain Malmain
99a156f783
Implement From LibAFL Error for Qemu Error (#2641)
* impl from qemu error for libafl error

* new error for frontends like qemu or frida

* exhaustive match

* frontend -> runtime
2024-11-03 21:09:16 +01:00
Valentin Huber
d4fbe1754f
Move ShMem persisting flag to a new constructor (#2649)
* moving shmem persisting to take an owned value, adding test

* clean code updates

* adding imports conditionally

* fixing tests

* moving persistent mmap shmem to custom constructor

* excluding miri properly

* fixing formatting
2024-11-03 03:13:10 +01:00
Marco C.
89cff63702
Add taplo to pre-commit (#2646) 2024-10-31 17:55:10 +01:00
Romain Malmain
6d55626a48
libafl_qemu: update qemu to v9.1.1 (#2636)
* update qemu to v9.1.1

* adapting stuff to qemu 9.1

* fix for new qemu gen_callN and x86 decoder

* remove outdated qemu configuration option
2024-10-31 17:54:37 +01:00
Dominik Maier
c86e116d9a
Remove serde_json dependency from libafl_bolts (#2639)
* Remove serde_json dependency from libafl_bolts

* more like a serialize err

* Fix nautilus json
2024-10-31 14:16:53 +01:00
Romain Malmain
47120834dc
libafl_qemu: do not test slirp in ci for usermode (#2644) 2024-10-30 17:47:37 +01:00
Romain Malmain
9da113e7a8
Fix RISC-V port issues (#2642)
fix riscv{32,64} stuff
2024-10-30 14:10:50 +01:00
s1341
af06d75d3e
frida: bump version (#2640)
* frida: bump version

* frida_windows: fix
2024-10-30 12:31:40 +01:00
saibotk
83c87acd5b
libafl_qemu: Add RISCV support (#2367)
* libafl_qemu: Add RISCV support

Adds the following targets (as features):
- riscv32
- riscv64

Added `RISCVCPU` and `CPURISCVState` to the bindings allow list.

Added riscv.rs to the arch module, with all necessary functions and
registers implemented and mapped.
The registers are the same as the ones found in qemus gdbstub xml found
after a build.

Additionally we added all syscall numbers for riscv 64 bit (already
supported by the `syscall_numbers` crate) and also added the missing
ones for riscv 32 bit. We compared both lists and their differences /
equalities with a simple python script and generated a list of the
missing ones, to be complete.
We might PR those to the `syscall_numbers` crate later on.

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-10-30 10:33:03 +01:00
Dominik Maier
6eb2dafd34
Feature: Make executors and feedbacks easier to use outside of the fuzzing loop (extends #2511) (#2637)
* feat(libafl_core): make executors and feedbacks more cleanly usable outside of LibAFLs Fuzzer loop

* cargo +nightly fmt

* updated type constraints

* reformatted and final type constraint fixes

* made unicode extraction stage useful separately

* fix libafl_cc error message

* fix state type constraint to be constrained on the method

* removed unnecessary observer constraint

* renamed unused variables

* fix unnecessary error wrapping in helper functions

* converted unicode conversion stage into associated function and fixed nautilus changes

* more update

* Remove extra I

* more fmt

* bounds?

* less bounds

* more less bounds

* different trait bounds again

* more less generics

* fix unicode

* fix list

* remove unneeded bound

---------

Co-authored-by: Lukas Dresel <Lukas-Dresel@users.noreply.github.com>
Co-authored-by: Toka <tokazerkje@outlook.com>
2024-10-29 18:35:17 +01:00
Aarnav
0f744a3abb
Add Python Grammar Loader for Nautilus (#2635)
* add python grammar loader for Nautilus

* fmt

* fmt toml

* add python to macos CI deps

* install python

* fmt

* ci

* clippy

* fix workflow

* fmt

* fix baby nautilus

* fix nautilus sync

* fmt

* fmt

* clippy

* typo

* fix miri

* remove pyo3 from workspace to packages which need it and make it optional

* go back to AsRef<Path> for nautilus grammar loading

* replace hardcoded python flags for macos build

* typo

* taplo fmt

* revert formatting of libafl_qemu_arch

* ci

* typo

* remove expects in NautilusContext::from_file and make them Results

* remove not(miri) clause in test

* try and fix python build fir ios and android

* again

* android

* tmate

* fix android build

* document load_python_grammar

* log if python or json when loading nautilus grammar

* make nautilus optional

* add nautilus as feature to forkserver_simple_nautilus
2024-10-29 11:32:59 +01:00
Aarnav
58fad2befd
libafl-fuzz: separate frida build + cmplog debug (#2591)
* libafl-fuzz: separate frida build

* cmplog debug

* update

* merge AflStatsStage
move time_tracker stage to LibAFL

* mandate track_hit_feedbacks feature for AflStatsStage

* afl_stats do not hardcode TimeoutFeedback and CrashFeedback names

* typo

* typo

* fix generics order

* add verify timeouts stage

* libafl: introduce set_timeout func to dynamically set timeouts for executor
libafl-fuzz: add verify_timeout stage

* add missing set_timeout implementations

* libafl-fuzz: move set_timeout and timeout from Executor to HasTimeout

* libafl-fuzz: add removed gitignore

* remove timeout from libafl_nyx::Executor and move it to NyxHelper

* clippy

* fix HasTimeout for QemuExecutor

* libafl-fuzz: remove observer handle usage in verify_timeouts
misc: remove prelude imports

* libafl-fuzz: fix foreign_sync_dirs option

* fmt && clippy

* clippy && fmt

* missing doc

* clippy

* bruh

* damned doc build

* trait fix

* impl HasTimeout for InProcessExecutor only if std

* clippy

* fix typo

* fix nostd build

* clippy

* remove most HasTimeout implementations for now

* typo

* remove redundant import

* misc

* fmt

* simplify trait bounds

* add old AflStatsStage back and rename it to StatsStage

* fix ci

* make set_timeout and timeout of HasTimeout inline

* fmt

* add gitignore

* serde_any fix

* tmate

* misc

* remove tmate

* test

* coordinate between capture_timeout and verify_timeout

* makefile

* fix

* fix

* fmt

* increase cmplog timeout

* semantic

* debug

* debug

* remove dbeug

* only test libafl-fuzz on CI for now

* better seed for cmplog?

* remove preflight check for now

* set Input type in forkserver

* debug

* tmate

* fix capture_timeout

* revert workflow

* run only libafl-fuzz

* remove pre-flight

* re-enable fuzzers on CI

* move capture_timeouts and verify_timeouts to main lib

* run fmt

* add note for verify timeouts

* add note in verify timeouts stage

* typo

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-10-28 11:40:04 +01:00
Dominik Maier
42b306a39f
Fix LibAFL-fuzz build (#2634) 2024-10-26 02:03:22 +02:00