227 Commits

Author SHA1 Message Date
Romain Malmain
94fa4014ac
Update pyo3 to version 0.23.2 (#2732)
* update pyo3 to latest version

* add python bindings to workspace

* make pyo3 stuff dependent of workspace again

* adapt implementation for the newest version of pyo3
2024-11-27 19:01:31 +01:00
Valentin Huber
e9af21cf71
Adding function to manually name a Mmap ShMem (#2729)
* adding function to manually name a mmap shmem

* fixing tests

* making code more readable

* Add MIGRATION.md (#2727)

* updating MIGRATION.md

* removing unnecessary assignment

* fixing typo in MIGRATION.md

* fixing formatting

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
2024-11-26 20:08:31 +01:00
Mrmaxmeier
6ce447ee17
Specify minimum supported rust version and check in CI (#2726) 2024-11-26 17:58:01 +01:00
Addison Crump
e53dd4e6cc
Fixup serdeany_autoreg (#2721)
* fixup serdeany_autoreg

* missed a spot

* remove explicit checks in ps1 as this is set by Cargo.toml
2024-11-24 03:00:54 -03:00
Dominik Maier
f3ab3c744e
Bump ctor dependency to make nightly compile again (#2713) 2024-11-20 18:18:38 -03:00
David CARLIER
f74a965ead
bolts: fix build for tiers 3 platforms. (#2700)
cater to platforms knowingly support this feature instead.
2024-11-18 13:36:41 -03:00
Marco C.
eb2ac10d1e
Libafl workspace internal deps in workspace Cargo.toml (#2691)
* Add internal deps to workspace

* libafl: use workspace internal deps

* libafl_bolts: use workspace internal deps

* 0.14.0

* use workspace internal deps
2024-11-15 18:13:39 +01:00
Andrea Fioraldi
d339a58f7c
Bump to 0.14.0 (#2692) 2024-11-13 23:10:35 +01:00
Dominik Maier
7938acc4ce
Actually make ConstMapObserver work, introduce nonnull_raw_mut macro (#2687)
* Actually make ConstMapObserver work

* fixes

* does that work?

* mas
2024-11-13 00:35:46 -03:00
Aarnav
87f5f21e01
libafl-fuzz: introduce nyx_mode (#2503)
* add nyx_mode

* fix frida ci?

* damn clippy

* clippy
2024-11-12 22:51:07 -03:00
Marco C.
f7f8dff6cd
Add Intel PT tracing support (#2471)
* WIP: IntelPT qemu systemmode

* use perf-event-open-sys instead of bindgen

* intelPT Add enable and disable tracing, add test

* Use static_assertions crate

* Fix volatiles, finish test

* Add Intel PT availability check

* Use LibAFL errors in Result

* Improve filtering

* Add KVM pt_mode check

* move static_assertions use

* Check for perf_event_open support

* Add (empty) IntelPT module

* Add IntelPTModule POC

* partial ideas to implement intel pt

* forgot smth

* trace decoding draft

* add libipt decoder

* use cpuid instead of reading /proc/cpuinfo

* investigating nondeterministic behaviour

* intel_pt module add thread creation hook

* Fully identify deps versions

Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates

* Move mem image to module, output to file for debug

* fixup! Use static_assertions crate

* Exclude host kernel from traces

* Bump libipt-rs

* Callback to get memory as an alterantive to image

* WIP Add bootloader fuzzer example

* Split availability check: add availability_with_qemu

* Move IntelPT to observer

* Improve test docs

* Clippy happy now

* Taplo happy now

* Add IntelPTObserver boilerplate

* Hook instead of Observer

* Clippy & Taplo

* Add psb_freq setting

* Extremely bad and dirty babyfuzzer stealing

* Use thread local cell instead of mutex

* Try a trace diff based naive feedback

* fix perf aux buffer wrap handling

* Use f64 for feedback score

* Fix clippy for cargo test

* Add config format tests

* WIP intelpt babyfuzzer with fork

* Fix not wrapped tail offset in split buffer

* Baby PT with raw traces diff working

* Cache nr_filters

* Use Lazy_lock for perf_type

* Add baby_fuzzer_intel_pt

* restore baby fuzzer

* baby_fuzzer with block decoder

* instruction decoder instead of block

* Fix after upstream merge

* OwnedRefMut instead of Cow

* Read mem directly instead of going through files

* Fix cache lifetime and tail update

* clippy

* Taplo

* Compile caps only on linux

* clippy

* Fail compilation on unsupported OSes

* Add baby_fuzzer_intel_pt to CI

* Cleanup

* Move intel pt + linux check

* fix baby pt

* rollback forkexecutor

* Remove unused dep

* Cleanup

* Lints

* Compute an edge id instead of using only block ip

* Binary only intelPT POC

* put linux specific code behind target_os=linux

* Clippy & Taplo

* fix CI

* Disable relocation

* No unwrap in decode

* No expect in decode

* Better logging, smaller aux buffer

* add IntelPTBuilder

* some lints

* Add exclude_hv config

* Per CPU tracing and inheritance

* Parametrize buffer size

* Try not to break commandExecutor API pt.1

* Try not to break commandExecutor API pt.2

* Try not to break commandExecutor API pt.3

* fix baby PT

* Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620)

* support (unsafe) on_crash / on_timeout callbacks for modules

* use libc types in bindgen

* Move common code to bolts

* Cleanup

* Revert changes to backtrace_baby_fuzzers/command_executor

* Move intel_pt in one file

* Use workspace deps

* add nr_addr_filter fallback

* Cleaning

* Improve decode

* Clippy

* Improve errors and docs

* Impl from<PtError> for libafl::Error

* Merge hooks

* Docs

* Clean command executor

* fix baby PT

* fix baby PT warnings

* decoder fills the map with no vec alloc

* WIP command executor intel PT

* filter_map() instead of filter().map()

* fix docs

* fix windows?

* Baby lints

* Small cleanings

* Use personality to disable ASLR at runtime

* Fix nix dep

* Use prc-maps in babyfuzzer

* working ET_DYN elf

* Cleanup Cargo.toml

* Clean command executor

* introduce PtraceCommandConfigurator

* Fix clippy & taplo

* input via stdin

* libipt as workspace dep

* Check kernel version

* support Arg input location

* Reorder stuff

* File input

* timeout support for PtraceExec

* Lints

* Move out method not needing self form IntelPT

* unimplemented

* Lints

* Move intel_pt_baby_fuzzer

* Move intel_pt_command_executor

* Document the need for smp_rmb

* Better comment

* Readme and Makefile.toml instead of build.rs

* Move out from libafl_bolts to libafl_intelpt

* Fix hooks

* (Almost) fix intel_pt command exec

* fix intel_pt command exec debug

* Fix baby_fuzzer

* &raw over addr_of!

* cfg(target_os = "linux")

* bolts Cargo.toml leftover

* minimum wage README.md

* extract join_split_trace from decode

* extract decode_block from decode

* add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0

* More generic hook

* fix windows

* Update CI, fmt

* No bitbybit

* Fix docker?

* Fix Apple silicon?

* Use old libipt from crates.io

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-11-12 22:34:46 -03:00
Marco C.
f3aa88b400
Use version.workspace (#2682) 2024-11-12 15:45:35 +01:00
Dominik Maier
8617fa6603
Introduce workspace (again) (#2673)
* Trying to redo workspace deps again after #2672

* unused

* clippy
2024-11-09 19:11:01 +01:00
Dominik Maier
21f8b1d147
Replace addr_of with &raw across the codebase (#2669)
* Replace addr_of with &raw across the codebase

* fix fixes

* more fix

* undo clang fmt?

* oops

* fix?

* allocator fix

* more fix

* more more

* more docs

* more fix

* mas mas mas

* hm

* more

* fix Frida

* needed

* more error

* qemu
2024-11-08 17:19:55 +01:00
bernhl
cfe124043e
Add memfd shmem backend (#2647) 2024-11-06 11:37:43 +01:00
Dominik Maier
b5c9bffe50
Core::errors::Error is stable now (#2664) 2024-11-05 16:48:31 +01:00
Dongjia "toka" Zhang
e0c5304e32
Don't use workspace dependencies (#2652)
* rev

* postcard

* serde

* fmt

* a

* dependabot

* postcard again

* hashbrown

* upd

* add

* serde?
2024-11-04 17:47:40 +01:00
Romain Malmain
49ea0b03a6
Compile-time edge module compilation check, native support for ConstMapObserver (#2592)
* compile-time edge module compilation trick

* clippy

* possible since rust 1.79

* split edge module in submodules

* Update frida to 0.14.0 (#2596)

* update frida crate to the latest version

* adapt libafl_frida to the latest version of frida

* tracers and generators private modules

* do not use star export.

* same for drcov

* forgot a file...

* first draft of generic-based edge module for ConstantLengthMapObserver.

* integration of OwnedSizedSlice.

replaced OwnedSlice in ConstMapObserver by the new OwnedSizedSlice.

* fix serde stuff

* no std

* import

* fixed qemu_cmin with new constant map abstraction.

* fix const map

* fix clippy from another pr...

* fix non-null usage

* fix ci?

* new feature stuff

* fixes

* minor fixes

* fmt

* non null

* im stupid

* fmt

* fix fuzzer

* fix fuzzers

* sized slice

* fuzzer fixes

* ptr::NonNull -> NonNull

* shorter trait length

* fmt
2024-11-04 14:34:52 +01:00
Romain Malmain
99a156f783
Implement From LibAFL Error for Qemu Error (#2641)
* impl from qemu error for libafl error

* new error for frontends like qemu or frida

* exhaustive match

* frontend -> runtime
2024-11-03 21:09:16 +01:00
Valentin Huber
d4fbe1754f
Move ShMem persisting flag to a new constructor (#2649)
* moving shmem persisting to take an owned value, adding test

* clean code updates

* adding imports conditionally

* fixing tests

* moving persistent mmap shmem to custom constructor

* excluding miri properly

* fixing formatting
2024-11-03 03:13:10 +01:00
Dominik Maier
c86e116d9a
Remove serde_json dependency from libafl_bolts (#2639)
* Remove serde_json dependency from libafl_bolts

* more like a serialize err

* Fix nautilus json
2024-10-31 14:16:53 +01:00
Aarnav
0f744a3abb
Add Python Grammar Loader for Nautilus (#2635)
* add python grammar loader for Nautilus

* fmt

* fmt toml

* add python to macos CI deps

* install python

* fmt

* ci

* clippy

* fix workflow

* fmt

* fix baby nautilus

* fix nautilus sync

* fmt

* fmt

* clippy

* typo

* fix miri

* remove pyo3 from workspace to packages which need it and make it optional

* go back to AsRef<Path> for nautilus grammar loading

* replace hardcoded python flags for macos build

* typo

* taplo fmt

* revert formatting of libafl_qemu_arch

* ci

* typo

* remove expects in NautilusContext::from_file and make them Results

* remove not(miri) clause in test

* try and fix python build fir ios and android

* again

* android

* tmate

* fix android build

* document load_python_grammar

* log if python or json when loading nautilus grammar

* make nautilus optional

* add nautilus as feature to forkserver_simple_nautilus
2024-10-29 11:32:59 +01:00
WorksButNotTested
03af6aaf0c
Frida scripting support (#2506)
Co-authored-by: Your Name <you@example.com>
2024-10-25 17:18:25 +02:00
Langston Barrett
dfd5609c10
bolts: Simplify definition of nonzero! macro (#2624)
* bolts: Simplify definition of `nonzero!` macro

* Non-Usize NonZero

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-10-24 00:28:47 +02:00
Dongjia "toka" Zhang
fda1596ee2
Use nonzero! macro (#2589)
* nonzero_lit

* nonzero

* std instead of core

* a

* l

* test

* import

* api

* api

* aaaaa

* apiapi

* api

* api

* api

* mm

* api

* non zero

* FMT

* pls

* nnnaaasdfadsfafdsa

* pls

* MM

* fix

* a

* sat add

* aa

* mistake

* unreachable

* no generic

* api change

* a
2024-10-18 23:33:03 +02:00
Romain Malmain
d48a7d508d
Centralize clippy lints in workspace (#2606)
* centralize clippy definition

* fmt

* add update bindings script

* add a checked and unchecked version of memory read to qemu stuff

also, a lot of clippy thing

* update binding position

* rm old script, new one is a bit better

* update doc

* macos clippy

* adapt fuzzers

* windows clippy

* fix fuzzer

* windows clippy

* remove old allowed clippy

* remove some allowed clippy

* use default features for serde_json in gramatron

* better error handler for failed rw to memory
2024-10-15 13:31:01 +02:00
Dominik Maier
cad9e2717e
Remove prelude from default features (#2608)
* Remove prelude from default features

* Prelude

* something
2024-10-12 13:16:22 +02:00
Dongjia "toka" Zhang
c12c6f31e2
Change fuzzbench_qemu fuzzer (#2520)
* change fuzzbench_qemu

* real test

* fix qemu crash hook

* update bindings

* fix fork executor, reduce trait bound overhead

* make EdgeModule depend on observer to get ptrs.

* do not make EdgeCoverageModule::new public

* map observer as builder call

* adapt examples with new edge coverage module builder.

* TMP: everyone is a variable length map observer

* reuse profile path script

* fix absolute paths

* remove some dependencies to make pipeline faster

* compile-time builder initialization check

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
2024-10-08 15:18:13 +02:00
Romain Malmain
1fbb18cb99
Versioning unification, dependencies update, logging optimization (#2560)
* versioning unification: use x.y.z format everywhere

* do not compile low-level logs (< info level) by default in fuzzers

* update dependencies to the latest versions

* add members to workspace.

* use workspace for common dependencies

* add vscode native support

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-10-07 12:03:29 +02:00
Dominik Maier
4fc136cd1c
Rand below should take a NonZero parameter (#2519)
* Rand below should take a NonZero parameter

* More

* more

* More

* fix build

* bit of clippy

* more clippy

* more clippy

* More clippy

* More more

* more nonzero

* fix multipart

* Cleanup, more unsafe

* fix

* fix unicode

* clippy, fmt

* more

* More safer and more better

* MaxStackPow

* fix merge fails

* make random_slize_size faster

* fix

* more

* fix
2024-10-04 02:16:10 +02:00
Romain Malmain
9a50868058
Discard non-new testcase events for multi-machine messages (#2583)
* discard non-new testcase events

* clippy
2024-10-03 18:46:26 +02:00
Marco C.
9b0eccf73b
Do not require Sized for OwnedRefMut as_ref/as_mut (#2581) 2024-10-03 13:35:00 +02:00
Dominik Maier
82110472d6
Mark unsafe functions unsafe, as Ferris intended (#2559)
* Mark unsafe functions unsafe, as Ferris inteded

* More

* more safety?

* more fix

* actually safe

* More cleanup

* More fix

* more unsafe

* fix imports

* more unsafe

* fixes

* bring back the memories
2024-09-28 16:46:39 +02:00
Dongjia "toka" Zhang
79b8390214
No Uses* for Corpus, Mutators (#2547)
* wip

* this shit is crazy

* from windows

* fmt

* fuck

* fmt

* aa

* reduce generics for generator

* do stuff for mutators, too

* aa

* a

* fmt

* idk

* getting things compiling

* fix merge

* idk

* fuzzers

* nire

* more

* a

* frida win

* cmin

* mm

* qemu?

* del ubuntu dep

---------

Co-authored-by: Your Name <you@example.com>
2024-09-26 20:03:03 +02:00
Romain Malmain
c944a70056
Linux kernel fuzzing example (#2496)
* linux kernel (x509_cert) and process fuzzing example

* rework filters

* update to latest qemu

* working for process and kernel fuzzing

* new i2s mutator for binary only fuzzers

* refactoring modules with new filtering interface

* add state as parameter of harness

* hide unused global in usermode

* Script for stub bindings generation

* do not try to check whether it is worth generating the bindings, always
  generate when the env variable is on.

* add taplo to fmt_all.sh

* Moved fuzzers (again) in a target-centric way.

* fix rust 2024 warnings.

* new libafl_qemu harness structure.

* rename qemu_systemmode into qemu_baremetal

* fix qemu baremetal makefile

* fix formatter

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-09-26 14:29:33 +02:00
Andrea Fioraldi
72893797b4
Fix panic in mmap shmem when full_file_name is less than MAX_MMAP_FILENAME_LEN (#2536)
* Fix panic in mmap shmem

* duh

* clippy

* fix null bytes

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-09-24 03:37:03 +02:00
Dominik Maier
f0d85aaaa5
No more (direct) mutable references to mutable statics (#2550)
* No more (direct) mutable references to mutable statics

* More

* Fix build, but it's unsafe dude

* more

* fmt

* More fix

* more?
2024-09-24 03:29:49 +02:00
Dominik Maier
8ccff4b77f
Add StdState::nop() for simple state creation, make CI happy again on latest nightly. (#2521)
* Make CI happy again on latest nightly

* Fix build

* Unneeded comment

* fix tests

* clippy
2024-09-12 00:33:55 +02:00
Romain Malmain
203d3d340a
Multi machine follow-up (#2334)
* improved tree generator.
possibility to dump state on exit.

* warnings.

* clippy

* clippy

* aaaaaa

* aaaaaa

* make nix mandatory for unix

* stddddddd

* stddddddd

* stddddddd

* aaa

* clippy

* doc

* aaa

* aaa

* aaa

* aaa

* fix

* aaaaaaaa

* dump state

* aaa

* aaa

* aaa

* more minimal dump

* aaa

* aaa

* aaa

* simpler tc dump

* pub

* more pub

* revert a bit

* release by default

* delete Makefile.toml

* release

* dump execs

* merge

* delete stuff

* aa

* ff

* dig

* FMT cargo stuf

---------

Co-authored-by: Toka <tokazerkje@outlook.com>
2024-09-04 18:42:54 +02:00
Dominik Maier
e2cc78f274
Better documentation headers (clippy) (#2501)
* Better documentation headers (clippy)

* more doc

* more fixes

* Even more

* more

* even more

* concrete

* fmt

* even more more

* tiny typo

* more

* more

* More

* more

* more docs?

* more docs
2024-08-26 16:31:22 +02:00
Marco C.
a388012429
Do not use caret requirements (#2498) 2024-08-24 00:02:37 -05:00
Romain Malmain
1f55458293
Fix fmt & remaining md links (#2497) 2024-08-22 19:08:44 -05:00
Alexander Qi
0018f7e406
bolts: Support dump_registers on Windows/x86 and Windows/aarch64 and fix sig_ign on Windows/x86 (#2494)
* bolts: Support dump_registers on Windows/x86

* bolts: Support dump_registers on Windows/aarch64

* bolts: Fix sig_ign() on Windows/x86

* bolts: Fix format of dump_registers under Windows

* bolts: Add test for dump_register under Windows
2024-08-22 18:22:24 -05:00
Romain Malmain
8d0ad23856
Check markdown links validity in CI (#2495)
* add markdown link check in ci

* fix broken links when possible.
configuration file for md checker.

* accept code 200...

* more fixes

* more fixes

* more fixes
2024-08-22 18:15:15 -05:00
Romain Malmain
4b87d7f4eb
Custom QEMU emulator typed builder + ExitHandler / Commands refactoring (#2486)
* Add a custom typed builder for Emulator

* Unify qemu_init for usermode and systemmode

* Remove env from qemu init args (it is unused in QEMU in practice)

* expose thread hooks to systemmode

* rename qemu_config to config

* Replace ExitHandler by EmulatorDriver

* Reorder generics alphabetically for Qemu{,Fork}Executor

* Moved snapshot manager to Emulator to continue centralizing mains objects in the same structure

* Reimplementation of CommandManager working with enums instead of tables

* Macro has been adapted to do this work automatically

* Moved snapshot stuff to dedicated module

* Removed many Rc<RefCell<...>>, now useless with the removal of vtables

* Builder given by Emulator via `Emulator::builder`. Reduced trait bound overhead
2024-08-21 16:36:45 +02:00
David CARLIER
291fdeb85e
bolts: fix openbsd build (#2467)
Co-authored-by: dcarlier <dcarlier@openbsdvbox.chello.ie>
2024-08-01 08:14:09 +02:00
Romain Malmain
c319fe2033
Cheap reader for bytes slice (#2261)
* initial draft implementation

* change name to follow rust convention.

* revert use of HasTargetBytes instead of HasMutatorBytes for BytesSubInputMut

* clippy

* nostd

* clippy

* clippy

* * HasLen required if implementing HasTargetBytes.
* Added a checked version of the read to slice.

* clippy

* fix name.
better doc.

* added a common bytes trait for HasTargetBytes and HasMutatorBytes.

* change interface

* fix tests

* clippers

* use byte slice for subbytes

* adapt to main

* fix doc

* mut sub slice version. return subinput to old state, and add subslice stubs

* better api, doc fixes.

* Don't clone, reshuffle

* Move and rename

* Uh-oh

* move to bolts. rename things.

* nostd

* format

* alloc

* fix doc

---------

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
2024-07-30 13:46:40 +02:00
Dominik Maier
c4c0fb6750
Update pyo3 (#2459)
* Update pyo3

* Update pyo3 to 0.22

* Fix qemu python

* clippy

* fmt

* nautilus python

* More pyo3

* Make signature more legible
2024-07-29 22:55:28 +02:00
David CARLIER
bab50f23c5
bolts: fix build warning with missing doc comment for haiku (#2455) 2024-07-28 22:21:06 +02:00
Mrmaxmeier
afbdc02c54
Format all tomls with taplo fmt (#2436)
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-07-26 10:03:10 +02:00