1162 Commits

Author SHA1 Message Date
0a233aad48 plot stgsize + observe_hitcounts-- 2024-11-14 13:22:04 +01:00
Andrea Fioraldi
d339a58f7c
Bump to 0.14.0 (#2692) 2024-11-13 23:10:35 +01:00
Aarnav
d334860148
Feature: libafl-fuzzfuzzbench (#2689)
* fuzzbench

* clippy

* fmt

* fix unicorn CI?
2024-11-13 13:32:21 +01:00
Dominik Maier
7938acc4ce
Actually make ConstMapObserver work, introduce nonnull_raw_mut macro (#2687)
* Actually make ConstMapObserver work

* fixes

* does that work?

* mas
2024-11-13 00:35:46 -03:00
Dongjia "toka" Zhang
d5db2c0e3f
LibAFL: Remove tui_monitor from default features (#2685)
* No Usermode default

* no tui

* gg

* try fix CI

* fmt

---------

Co-authored-by: Dominik Maier <dmnk@google.com>
2024-11-12 22:55:02 -03:00
Aarnav
87f5f21e01
libafl-fuzz: introduce nyx_mode (#2503)
* add nyx_mode

* fix frida ci?

* damn clippy

* clippy
2024-11-12 22:51:07 -03:00
Marco C.
f7f8dff6cd
Add Intel PT tracing support (#2471)
* WIP: IntelPT qemu systemmode

* use perf-event-open-sys instead of bindgen

* intelPT Add enable and disable tracing, add test

* Use static_assertions crate

* Fix volatiles, finish test

* Add Intel PT availability check

* Use LibAFL errors in Result

* Improve filtering

* Add KVM pt_mode check

* move static_assertions use

* Check for perf_event_open support

* Add (empty) IntelPT module

* Add IntelPTModule POC

* partial ideas to implement intel pt

* forgot smth

* trace decoding draft

* add libipt decoder

* use cpuid instead of reading /proc/cpuinfo

* investigating nondeterministic behaviour

* intel_pt module add thread creation hook

* Fully identify deps versions

Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates

* Move mem image to module, output to file for debug

* fixup! Use static_assertions crate

* Exclude host kernel from traces

* Bump libipt-rs

* Callback to get memory as an alterantive to image

* WIP Add bootloader fuzzer example

* Split availability check: add availability_with_qemu

* Move IntelPT to observer

* Improve test docs

* Clippy happy now

* Taplo happy now

* Add IntelPTObserver boilerplate

* Hook instead of Observer

* Clippy & Taplo

* Add psb_freq setting

* Extremely bad and dirty babyfuzzer stealing

* Use thread local cell instead of mutex

* Try a trace diff based naive feedback

* fix perf aux buffer wrap handling

* Use f64 for feedback score

* Fix clippy for cargo test

* Add config format tests

* WIP intelpt babyfuzzer with fork

* Fix not wrapped tail offset in split buffer

* Baby PT with raw traces diff working

* Cache nr_filters

* Use Lazy_lock for perf_type

* Add baby_fuzzer_intel_pt

* restore baby fuzzer

* baby_fuzzer with block decoder

* instruction decoder instead of block

* Fix after upstream merge

* OwnedRefMut instead of Cow

* Read mem directly instead of going through files

* Fix cache lifetime and tail update

* clippy

* Taplo

* Compile caps only on linux

* clippy

* Fail compilation on unsupported OSes

* Add baby_fuzzer_intel_pt to CI

* Cleanup

* Move intel pt + linux check

* fix baby pt

* rollback forkexecutor

* Remove unused dep

* Cleanup

* Lints

* Compute an edge id instead of using only block ip

* Binary only intelPT POC

* put linux specific code behind target_os=linux

* Clippy & Taplo

* fix CI

* Disable relocation

* No unwrap in decode

* No expect in decode

* Better logging, smaller aux buffer

* add IntelPTBuilder

* some lints

* Add exclude_hv config

* Per CPU tracing and inheritance

* Parametrize buffer size

* Try not to break commandExecutor API pt.1

* Try not to break commandExecutor API pt.2

* Try not to break commandExecutor API pt.3

* fix baby PT

* Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620)

* support (unsafe) on_crash / on_timeout callbacks for modules

* use libc types in bindgen

* Move common code to bolts

* Cleanup

* Revert changes to backtrace_baby_fuzzers/command_executor

* Move intel_pt in one file

* Use workspace deps

* add nr_addr_filter fallback

* Cleaning

* Improve decode

* Clippy

* Improve errors and docs

* Impl from<PtError> for libafl::Error

* Merge hooks

* Docs

* Clean command executor

* fix baby PT

* fix baby PT warnings

* decoder fills the map with no vec alloc

* WIP command executor intel PT

* filter_map() instead of filter().map()

* fix docs

* fix windows?

* Baby lints

* Small cleanings

* Use personality to disable ASLR at runtime

* Fix nix dep

* Use prc-maps in babyfuzzer

* working ET_DYN elf

* Cleanup Cargo.toml

* Clean command executor

* introduce PtraceCommandConfigurator

* Fix clippy & taplo

* input via stdin

* libipt as workspace dep

* Check kernel version

* support Arg input location

* Reorder stuff

* File input

* timeout support for PtraceExec

* Lints

* Move out method not needing self form IntelPT

* unimplemented

* Lints

* Move intel_pt_baby_fuzzer

* Move intel_pt_command_executor

* Document the need for smp_rmb

* Better comment

* Readme and Makefile.toml instead of build.rs

* Move out from libafl_bolts to libafl_intelpt

* Fix hooks

* (Almost) fix intel_pt command exec

* fix intel_pt command exec debug

* Fix baby_fuzzer

* &raw over addr_of!

* cfg(target_os = "linux")

* bolts Cargo.toml leftover

* minimum wage README.md

* extract join_split_trace from decode

* extract decode_block from decode

* add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0

* More generic hook

* fix windows

* Update CI, fmt

* No bitbybit

* Fix docker?

* Fix Apple silicon?

* Use old libipt from crates.io

---------

Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
2024-11-12 22:34:46 -03:00
Dominik Maier
0ef0684e43
LibAFL_QEMU: Don't return a generic Address from Register reads (#2681)
* LibAFL_QEMU: Make ReadReg always return GuestReg type

* Don't return a generic address

* fix fuzzers

* fix mips
2024-11-12 22:14:35 -03:00
Valentin Huber
ae9ab80ad9
Fixing version pinning in CI (#2679) 2024-11-12 00:49:58 +01:00
Subhojeet Mukherjee, PhD
e25094eb4f
Lower capped RAND generators (#2671)
* Lower capped rand generators

* Updated all references to RAND generators

* Formatting updates

* New RAND bytes generator constructor

* Revert "Updated all references to RAND generators"

This reverts commit 9daad894b25ec3867daf93c4fe67c03abec1d8c6.

* Revert "Formatting updates"

This reverts commit ff2a61a366c48b3f313878f62409e51b1e1ed663.

* cargo nightly format

* Added must_use to with_min_size
2024-11-11 19:36:55 +01:00
cd3c101e87 define critical benchmark sets 2024-11-11 15:45:36 +01:00
0c80801e9f define benchmark sets 2024-11-11 15:36:29 +01:00
2a61f51a69 keep more cases when pruning, try more non-favored cases 2024-11-11 13:50:44 +01:00
79f0aab769 config_stg_abbpath 2024-11-10 10:31:03 +01:00
b7710c7d8a rate-limit UserStats + tolerate pending notifications for 1ms 2024-11-10 10:21:36 +01:00
Valentin Huber
e32b3eae93
Introducing Launcher::overcommit, improving CI formatting (#2670)
* introducing Launcher::overcommit

* removing unnecessary cfg restrictions and clippy allows

* improving warning for wrong clang-format version

* installing black in the format CI

* Enforcing python formatting in CI

* extending formatting using black on all python files

* printing diff on black failure

* preferring python's black over system black

* moving to LLVM 19 for formatting
2024-11-09 19:13:51 +01:00
Dominik Maier
8617fa6603
Introduce workspace (again) (#2673)
* Trying to redo workspace deps again after #2672

* unused

* clippy
2024-11-09 19:11:01 +01:00
Dominik Maier
21f8b1d147
Replace addr_of with &raw across the codebase (#2669)
* Replace addr_of with &raw across the codebase

* fix fixes

* more fix

* undo clang fmt?

* oops

* fix?

* allocator fix

* more fix

* more more

* more docs

* more fix

* mas mas mas

* hm

* more

* fix Frida

* needed

* more error

* qemu
2024-11-08 17:19:55 +01:00
280025b505 add config_stg_aggregate 2024-11-08 16:10:26 +01:00
6dc55d6cc9 update all_bins 2024-11-08 16:04:14 +01:00
825d80b88d aggresive pruning 2024-11-08 16:00:23 +01:00
2bc7872a51 configure sched_stg_edge 2024-11-08 15:43:04 +01:00
ab6026535c rate-limit testcase printing 2024-11-08 11:05:36 +01:00
e6ec643781 fix release-detection for api -> isr -> app, fix crash on empty trace 2024-11-06 12:51:39 +01:00
Dominik Maier
36a24ab418
Custom Executor Example (#2570)
* [WIP] Custom Executor Example

* readme

* src/main.rs

* Finish

* fix warnings

* reame

* CI
2024-11-05 16:49:07 +01:00
Dhanvith Nayak
8de9dcaff7
Clean up clippy warnings in fuzzers/binary_only/* (#2662)
* clean clippy warnings from fuzzers/binary_only/*

* handle unused Results in fuzzers/binary_only/*

* format fuzzers/binary_only/qemu_cmin

* use unchecked memory write in qemu fuzzer examples

* create file_null in fuzzbench_fork_qemu
2024-11-05 15:22:14 +01:00
Dongjia "toka" Zhang
87e4a845e0
Reenable libfuzzer_stb_image_sugar test (#2659) 2024-11-04 22:46:12 +01:00
Dongjia "toka" Zhang
e0c5304e32
Don't use workspace dependencies (#2652)
* rev

* postcard

* serde

* fmt

* a

* dependabot

* postcard again

* hashbrown

* upd

* add

* serde?
2024-11-04 17:47:40 +01:00
jejuisland87654
1e0b5581e5
better definition for PROFILE_DIR in all Makefile.toml files (#2658) 2024-11-04 17:09:12 +01:00
6902e21c6a update demo-list 2024-11-04 16:27:49 +01:00
Romain Malmain
49ea0b03a6
Compile-time edge module compilation check, native support for ConstMapObserver (#2592)
* compile-time edge module compilation trick

* clippy

* possible since rust 1.79

* split edge module in submodules

* Update frida to 0.14.0 (#2596)

* update frida crate to the latest version

* adapt libafl_frida to the latest version of frida

* tracers and generators private modules

* do not use star export.

* same for drcov

* forgot a file...

* first draft of generic-based edge module for ConstantLengthMapObserver.

* integration of OwnedSizedSlice.

replaced OwnedSlice in ConstMapObserver by the new OwnedSizedSlice.

* fix serde stuff

* no std

* import

* fixed qemu_cmin with new constant map abstraction.

* fix const map

* fix clippy from another pr...

* fix non-null usage

* fix ci?

* new feature stuff

* fixes

* minor fixes

* fmt

* non null

* im stupid

* fmt

* fix fuzzer

* fix fuzzers

* sized slice

* fuzzer fixes

* ptr::NonNull -> NonNull

* shorter trait length

* fmt
2024-11-04 14:34:52 +01:00
328e762dd5 save per-instance crashes 2024-11-04 10:03:39 +01:00
ed90c12922 revert trace_job_response_times 2024-11-04 09:51:29 +01:00
660cf60fc5 skip error dumping 2024-11-02 11:52:56 +01:00
bc165eb094 update bench-config 2024-11-02 11:39:53 +01:00
79be3c8eca config 2024-11-02 11:37:29 +01:00
Romain Malmain
6d55626a48
libafl_qemu: update qemu to v9.1.1 (#2636)
* update qemu to v9.1.1

* adapting stuff to qemu 9.1

* fix for new qemu gen_callN and x86 decoder

* remove outdated qemu configuration option
2024-10-31 17:54:37 +01:00
e58d4ba6ff tune detection of irregular task instances 2024-10-30 16:18:12 +01:00
894c11935f display stats 2024-10-30 15:27:48 +01:00
7c71e683d8 don't remove all items from corpus 2024-10-30 14:45:32 +01:00
s1341
af06d75d3e
frida: bump version (#2640)
* frida: bump version

* frida_windows: fix
2024-10-30 12:31:40 +01:00
Dominik Maier
6eb2dafd34
Feature: Make executors and feedbacks easier to use outside of the fuzzing loop (extends #2511) (#2637)
* feat(libafl_core): make executors and feedbacks more cleanly usable outside of LibAFLs Fuzzer loop

* cargo +nightly fmt

* updated type constraints

* reformatted and final type constraint fixes

* made unicode extraction stage useful separately

* fix libafl_cc error message

* fix state type constraint to be constrained on the method

* removed unnecessary observer constraint

* renamed unused variables

* fix unnecessary error wrapping in helper functions

* converted unicode conversion stage into associated function and fixed nautilus changes

* more update

* Remove extra I

* more fmt

* bounds?

* less bounds

* more less bounds

* different trait bounds again

* more less generics

* fix unicode

* fix list

* remove unneeded bound

---------

Co-authored-by: Lukas Dresel <Lukas-Dresel@users.noreply.github.com>
Co-authored-by: Toka <tokazerkje@outlook.com>
2024-10-29 18:35:17 +01:00
0e7de2d109 imports 2024-10-29 18:23:02 +01:00
f1affaabff impl RemovableScheduler for GenerationScheduler 2024-10-29 18:21:46 +01:00
0a6888d06a add pruning message 2024-10-29 18:10:16 +01:00
d88cefb894 add experimental corpus pruning 2024-10-29 18:08:18 +01:00
4d0ec2e427 typo 2024-10-29 16:01:25 +01:00
a613156a32 micro fixes 2024-10-29 15:59:15 +01:00
013f3db487 implement STGSnippetStage, fix missing metadata 2024-10-29 14:07:52 +01:00
Aarnav
0f744a3abb
Add Python Grammar Loader for Nautilus (#2635)
* add python grammar loader for Nautilus

* fmt

* fmt toml

* add python to macos CI deps

* install python

* fmt

* ci

* clippy

* fix workflow

* fmt

* fix baby nautilus

* fix nautilus sync

* fmt

* fmt

* clippy

* typo

* fix miri

* remove pyo3 from workspace to packages which need it and make it optional

* go back to AsRef<Path> for nautilus grammar loading

* replace hardcoded python flags for macos build

* typo

* taplo fmt

* revert formatting of libafl_qemu_arch

* ci

* typo

* remove expects in NautilusContext::from_file and make them Results

* remove not(miri) clause in test

* try and fix python build fir ios and android

* again

* android

* tmate

* fix android build

* document load_python_grammar

* log if python or json when loading nautilus grammar

* make nautilus optional

* add nautilus as feature to forkserver_simple_nautilus
2024-10-29 11:32:59 +01:00