* Improve CI
* Fix markup errors
* Remove unnecessary matrix parameters
* Rename matrix jobs to tidy up the UI
* Allow the docker cache to be updated
* Fix cache name
* Share some caches
* Rename tools
* Add windows build script and additional changes to support windows for libafl-libfuzzer
* Update build scripts and harness wrapping directives
* Resolve issue with corpus edge count calculation
* Add help message and make fork do nothing on Windows
* Format harness_wrap.cpp
* Clippy happiness pass
* Clippy happiness pass
* Clippy happiness pass
* Correct logic
* Correct logic
* Update help output and make runs argument work
* Add test for libafl_libfuzzer on windows
* Add workflow for libafl_libfuzzer test
* Fix copy without dependent task
* Add libafl_libfuzzer_windows to preflight list
* Format harness
* Explicitly ignore windows fuzzer
* Remove windows-specific copy from unix instructions
* Ensure using nightly
* Fix job name
* Update build to use libFuzzer.lib on Windows to keep consistent with Linux
* Remove nightly requirement
---------
Co-authored-by: Rowan Hart <rowanhart@microsoft.com>
* fix ci
track: https://github.com/dtolnay/proc-macro2/pull/501
* format toml
* fix typo
* Use nightly for libafl_frida
* Install nightly for splitter
* Use dtolnay/rust-toolchain@nightly
* install ca-certificates
* Implement `Send` for `Shm`
it is safe because we take the ownership of the inner map pointer. Only potential
violation is deref the underlying pointer but that’s already unsafe. Therefore,
the properties of Send still hold within the safe world.
* Bump 1.87
* use std::io::pipe so that they are `Send`
* clippy
* upgrade
* Avoid phantomdata to make ForkserverExecutor !Send
* Missing gates
* Fix nostd
* bump in Dockerfile
* use dtolnay/rust-toolchain@stable instead
* setup latest toolchain on non Linux
* Fix typo
* stable ci
* fixer
* aa
* no -z flag
* doc
* ??
* this one needs
* nightly for some of them ..
* turn off fail fast for now 🥱
* aaa
* afasdfasfas
* mm
* tmate
* linker's fault
* fixer
* f
* dfsafdasfasf
* Make input loading fallible in SyncFromDiskStage
* fmt
* Add InvalidInput in Error enum and skip the Input in SyncFromDiskStage if it is encountered
* sync: remove file if error on loading in SyncFromDiskStage
* add reason to Error::InvalidInput
* sync make failure log a warning. clippy, fmt
* typo
* fmt
* fmt
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Fix fuzzers in docs, add them to CI (fixes#3185)
* ignore macos for now
* Ooops wrong one
* fix?
* clp
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* no more shellscript
* metadatas
* clp
* clippo
* this language breaks everyday
* stop
* 2404?
* purge rust
* everything is broken
* test
* ci dead
* aa
* aa
* remove broken shit
* remove broken shit
* broken shit
* a
* lol
* aaaaa
* AI fix
* aa
* aaa
* yes?
* a
* FK
* tmate
* wtf who tf did this shit????
* aa
* useless printlnt
* for system too
* unset first
* Add enable and disable methods for Corpus
* Add insert_inner_with_id to fix disable/enable & test
Since we need to insert an 'existing' testcase with a certain id, let's
use a private inner function for it.
It's not the most posh way to keep consistency, but as showed in the
test it works 'good enough'.
* Implement disable/enable for libafl_libfuzzer/corpus
* fix clippy issues and fix cfg[not"corpus_btreemap"]
* Move enable/disable from Corpus to a trait
* Rename HasCorpusEnablementOperations to EnableDisableCorpus
Unless we come up with a better idea. Naming is hard.
* fmt the changes
* Refactor to new forkserver
* Fix fuzzer examples and delete forkserver.c
* Fix clippy and doc warnings
* Fix symbol error
* Format Cargo.toml; Fix wrong doc link
* Fix silly typo.
* Rename ForkServer to Forkserver to make it more consistent
* Fix build.rs
* Merge StdForkserverParent and PersistentForkserverParent since the forkserver parent has not idea of whether it is persistent and the persistent version can handle the non-persistent version
* Fix clippy
* Do not take ownership for last_child_pid since it may be in persistent mode
* Split nautilus Python dependencies into separate feature
* Fix Nautilus imports order and tidy formatting
* Refactor tree generation to reduce code duplication
* Make plain_or_script_rule mutable
* librasan: Use bx instruction for ARM patch
* librasan: Support patching Thumb functions
* Get page size dynamically, protect two pages and undo changes after
* Rename ARM patch test functions
* librasan: Simplify patch for ARM
* Start on qemu_tmin
* WIP
* qemu_tmin working for single testcase.
Also some comment improvements.
* Add env_logger to baby_fuzzer
* Remove old reference.
* Added comment doc for qemu_tmin.
* Slight reorder for parallelisation
* Finished single-thread qemu_tmin
* Finishing touches to single thread version.
* A pre_commit.sh change I didn't notice.
* Duplicate to attempt multi-threaded version
* Fix taplo whine.
Mark "fork" mode as broken.
* Launcher for parallelisation implemented, but for one core.
* Running in parallel. Now need tidy up.
* Parallel version complete.
* Add comment
* Merged single-core/multi-core qemu_tmin into one crate
* Removed forkexecutor mode.
* Precommit fixes
* Add qemu_tmin to build_and_test.yml
* Clippy fixes
* Change tmin test cores to 0.
* Start removing non-apt deps
GH images come with baked-in software that doesn't
come from apt, getting the apt version will just
slow down CI since the baked in version will be
used anyway.
* Do not install llvm-X if already there
* Do not remove old llvm-X
Est-ce qu'ils nous derangent ?
* default llvm install is missing stuff
* CLang is already baked-in
* Set installed clang as default
* Move ARM deps to qemu fuzzers
* Introduce HasAflStyleTargetArguments and restore parse_afl_cmdline for CommandExecutor
* clippy
* Fix imports
* Move envs to the shared trait
* Move to a standalone file
* Format
* Gate via std and unix
* Fix
* clippy
* move InputLocation to afl_args
* clippy
* Fix
* fmt
* Fix more
* Shall last missing
* Rename to AflTargetArgs
* Fmt
* move AflTargetArgs to libafl_bolts and adapt CMainArgsBuilder to use it
* use_stdin is no longer needed
* leaveout
* Fix again
* Renaming
* Leave out lib.rs
* Leave out fmt
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Nop executor
* Qol types
* no stage
* A new() for NopStage
* clippy
* clippy again
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Fixing windows ci build in justfile. harness.cc modified for cl.exe. Additions to README.
* More precise compiler check for noinline definition.
---------
Co-authored-by: dkoch <dkoch@vigilantsys.com>
* Add rawmemchr
* Add stpncpy
* Add strchrnul
* Fix strcat
* Added strncat
* Add wcschr
* Minor tweak
* Add wcsncmp
* Add wcsnlen
* Add wcsrchr
* Add wmemchr
* Fix asan load/store sizes for wide string functions
* Refactor patches
* Rename tracking functions to prevent collision with allocator
* Change return type of asan_sym to make it consistent with the other native functions
* Fix mutex re-entrancy issue in Patches by splitting locks
* Fix tests on 32-bit platforms
---------
Co-authored-by: Your Name <you@example.com>
* POC
* POC
* pin the obj
* add map_input_to_memory
* still trying to get things to work
* Justfile
* lol
* nah bad idea
* done
* revert
* revert
* lol
* Move to libafl_qemu
* a
* add
* add
* lol
* clp
* a
* tmate
* Thank you bash I love you
* aaaaaaaa
* a
* bbb
* forkserver api
* poc
* i'm dumb
* add things
* use snapshot
* delete println debug
* anglais
* d
* fixer
* take care of further read
* take care about u32
* aa
* fix cursor
* mm
* pushing things temporary so i can try this path later
* delete useless setter
* rme
* BytesConverter
* now revert
* clp
* typo
* change how input passing works
* fuck
* fmt
* fixer
* fix
* lol
* lol
* lol
* disable CI
* delete assert
* clp
* a
* Added heap feature
* Rename feature and add some more docs
* Use document-features crate
* Expose the patching API for more flexibility
---------
Co-authored-by: Your Name <you@example.com>
* Fixes to main
* Add librasan
* Party like it's 2024
* Fix snapshot module to work with guest asan
* Fix guest_asan module
* Fixes to runner
* Fix linking issues using a REL
* Fix qemu_launcher
* Change modify_mapping to a method
* Fix gasan_test
* Remove debug from Justfile
* Optimize release build of librasan
* Set ulimit for qasan and gasan tests
* Tidy up symbol renaming
* Add missing symbols for PPC
* Change to support rustix 1.0.0
* Canonicalize the CUSTOM_ASAN_PATH
* Review changes
* Restructure backends
* release_max_level_info
* More review changes
* Clippy fixes
* Changes to reduce the burden on the CI
* Fix macos clippy
---------
Co-authored-by: Your Name <you@example.com>
* clean
* test thread 1
* add some error messages
* mm
* std
* mm
* aa
* pp
* fix
* change fuzzers
* fix
---------
Co-authored-by: Your Name <you@example.com>
* add share_objectives field to StdFuzzer
* uhhh maybe add field to StdState instead
* trivial
* implement to handle_in_client()
* fmt
* revert changes to state
* no gating on receiving objectives
* add query method to hasobjectives
* make input field of Event::Objective optional
* fmt and clippy
* move setter to hasobjectives
* better way to handle incoming objective
* fmt
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Fix implementation of UniqueTrace and add UnclassifiedTrace
* Update comments
* Move the implemetnation to classify_counts
* Only init for unique trace
* The missing inline
* Add a TODO
* Reference implementation of SAND: Decoupling Sanitization from Fuzzing for Low Overhead
* Format code
* make clippy happy
* Update docs
* clean output
* fmt
* Fix for nostd
* Update docs
* use use alloc::vec::Vec;
* Docs updates
* Update docs
* Format toml
* Format again
* Add CI
* Rename run_targets to run_target_all
* Update docs
* Update justfile to test fuzzer
* left out justfile
* Format
* Corner case when bitmap size is as small as 1
* Add comments
* clippy
* Format vuln.c
* Format toml
* Fix doc
* Fix justfile
* Move ExecutorsTuple to executors/mod.rs
* Fix
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* make fuzzer great again
* crash handlers
* hello from windows
* fk
* gee
* m
* temporary fix
* f
* mm
* CICI
* fixer
* Fix Dockerfile
* lol
* clp
* Fuck you clippy
* This lint makes no sense, 0
* ??
* a
* fix
* this lint makes 0 sense
* mm
* clp
* a
* a
* clp
* clippy
* clp
* mm
* FMT
* p
---------
Co-authored-by: Your Name <you@example.com>
Co-authored-by: toka <toka@tokas-MacBook-Air.local>
* Move all unnecessary std uses to core,alloc
* More
* more fix
* more
* more
* Remove libafl-fuzz grimoire
* more
* more
* more cleanup
* remove bins
* fix
* more fix
* Fix just
* Update dockerfile
* Changes to test all architectures for qemu-user fuzzers
* Update version of QEMU
* Fix qemu on aarch64
* Fix qemu on i386
---------
Co-authored-by: Your Name <you@example.com>
* Introduce ListInput
* Add remove mutators for ListInput
* Merge ListInput and MultipartInput
* Reimplement MultipartInput as a special case of ListInput
* Revert changes to Cargo.toml
* Add collection of generic listinput mutators
* Fix example
* Add note to MIGRATION
* Split list and multi into separate modules
* Fix docs
* Using string names again in the multi example fuzzer
* Remove unnecessary code
* Fix fuzzer
* Use key instead of name for MultipartInput key
* Prettier code in example fuzzer
* Do not convert slice to vec manually
* Decode with callback
* WIP restore intelpt module
* Fix build_target if target_dir doesn't exist
* WIP itelpt qemu/kvm example: bootloader
* qemu config refactoring
* Fix intel_pt_command_executor target dir
* * QEMU error refactoring*
* back to one QEMU init function
* other small things
* update test
* Bump libipt
* waitpid_filtered to ignore SIGWINCH
* Fix warnings unused manifest key: *.version
* Add export_raw feature to libafl_intelpt
* derive Debug for IntelPTHook
* Clippy
* Light refactor of EmulatorModules
* qemu is now a parameter to EmulatorModule callbacks and most function hooks.
* EmulatorModules is initialized before QEMU is initialized.
* Update target program ELF offsets
* fmt
* * asan fixed size accesses working with generics
* continue to propagate qemu argument as hook first parameter
* use pre_syscall* and post_syscall* everywhere
* fix some clippy stuff
* fmt
* Add comment to KVM pt_mode check
* refactor
* Add intel_pt_export_raw feature in libafl
* fix fuzzers
* * refactor asan and asanguest modules to avoid custom init of QEMU and use the module interface instead.
* adapt qemu_launcher example to fully work with emulator, since qemu must now be initialized by emulator.
* fmt
* clippy
* fix qemu_coverage
* fmt
* forgot qemu args in launcher
* map_error instead of unwrap
* use correct args
* Update to new libafl_qemu
* adapt api
* borrow checker friendly join_split_trace
and copy trace before deocde to prevent decoding failures
* testing stuff
* Set ip_filters (also) with builder
* Move trace to file
* Store a pt_builder in module
enable the setting of filters and other pt settings
* baby_bootloader target
* Best bootloader ever
* new builder?
* use closure for qemu config from emulator builder.
* better format
* clippy + fmt
* Fix build target
Create target directory if doesn't exist
* Remove filter on speculatively exec blocks
since also committed blocks can have this flag
* Add current ip_filters getter
* Fix possibile infinite loop in trace decode
* HW breakpoint + snapshot
* add snapshot and exit at first objective
* prefer raw pointers to slice_from_raw_parts_mut
since the latter is highly unsafe and allows more potentially dangerous reordering
* Add cpu option to QEMU config
* Add cpu option and minor improvements
* fix cargo run causing recompile
* no default devices
* windows clippy fix
* Exclude intel_pt feature from CI as all systemmode feats
* Add qemu_intel_pt_bootloader to CI
* Fix NopPageFilter
* Fix qemu_config
* Restore HW breakpoints
* Lints
* return Result for hw bp set/remove
* mark join_split_trace as unsafe
* Put the qcow2 in a tmpfs ramdisk
10x exec/sec
* Post merge fixes
* Try out libipt 0.3 alpha
* Try out libipt 0.3 alpha also in hook
* Clippy
* New libipt
* Post merge fixes
* Bump libipt
* Drive cache None
* Post merge fixes
* Use SectionInfo from libipt
* No slice::from_raw_parts_mut, just use raw pointer
* Cache the decoder builder
* Update qemu-bridge
* Add qemu -append param
* Move linux specific code to a mod, less #[cfg]s
* Add qemu initrd config
* Add qemu monitor tcp
* Add not enough ip filters message
* Fix wrong must_use
* Prevent possible infinite loop in block decoding in debug mode
* Clippy
* fix CI?
* Revert, keep libipt 0.3 and hw bp
---------
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Marco Cavenati <marco@lenovo300e>
* feat: unicorn support
* feat: update
* fix: remove unused files
* fix: clean
* fix: remove undeeded parameters
* fix: typo
* moved to justfile
* use log::debug!
* fix cargo and created justfile
* feat: add CI
* add runs on
* fix: CI
* fix: CI
* fix: don't use fork executor
* not needed anymore
* fix: CI
* fix: CI
* remove extra space
* Fixing the test_harness library name
* Fasan works, but testing of all features is pending
* Tests pass, before fixing clippy and fmt
* CLippy+fmt
* CLippy+fmt+tests running on linux
* Clippy
* Not stalkering the fuzzer. In the correct way
* Removing the instrumentation upon crash. Proper hooking of UnmapViewOfFile
* Fixes after the merge from the upstream (before 0.15.0). Still need to add the observer, clippy, fmt, and at least linux compilation
* Adding the helper observer and using it in the test
* Removing the observer from the wrong location
* Adapting to the new helper ownership model
* Adding an observer to shut down instrumentation upon crash
* Clippy + fmt
* Using mimalloc everywhere
* Deactivating before activating with the harness. Otherwise, gets stuck on Linux.
* Fixing imports for windows
* Using the new way of passing the handler
* Using frida_helper_shutdown_observer
* Clippy+fmt
* no-std, clippy
* Fmt
* Stable thread_id
* Clippy 18
* More clippy
* Formatting toml
* Fixing apples
* Fixing apples 2
* Fixing apples 3
* Upping to 0.16.7 (necessary for Windows)
* Clippy+fmt
* Enabling the allocator test after the fix and clarifying the importantce of the static runtime linking.
* Moving has_tls to bolts
* Proper handling of no-std, hopefully
* Another attempt to fix win no-std
* Not mine clippy complaint...
* Not mine clippy complaint #2...
* Dlmalloc not used, removing from dependencies
* Restoring target in config.toml (otherwise fails CI on Linux)
* lots of digging around, pray for us
* fixup?
* Revert "lots of digging around, pray for us"
This reverts commit 706c27201918e906e3401cd0d9e76546f889d1f5.
* Revert "fixup?"
This reverts commit 1d7c5d4fb5b1bd31f5e0c07492aa8ed64c6822f3.
* Revert artifact
* Revert fixups
* Removing unused
* Reverting to upstream/main
---------
Co-authored-by: Addison Crump <addison.crump@cispa.de>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* just port for binary only / systemmode fuzzers
* introduce just libraries, with pre-initialized variables and common recipes
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* drop grown address
* this clippy lint literally makes 0 sense!
* i hate you rust
* mm
* don't use drop! add comments for why alignment is not necessary
* move to just
* fuzzbench
* shell check
* version
* rme
* fixer
* fixing
* libpng friends
* pwd
* ps
* ps
* no fail fast.. for now
* windows
* l
* no powershell
* a
* fk
* fixer
* a
* Revert "fixer"
This reverts commit 76ae72d68686ead01ef914b658c5d8dac5aaee32.
* plzplz
* aaaa
* aa
* aa
* use absolute path, use LIBAFL_CC and LIBAFL_CXX
* why tabs????
* this job is not fun
* aa
* tmate debug
* disable cacheing
* del
* deldelg
* rename
* aaaa
* lol
* aaa
* lol
* lol
* 2nd tmate ...
* a
* lol
* lll
* shell ck
* please i'm about to cry
---------
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
* Remove uneeded Executor trait from handlers functions + set inproc_qemu_crash_handler to pub
* Add some documentation for QemuHooks
---------
Co-authored-by: celian <cglenaz>
* move to just
* fuzzbench
* shell check
* version
* rme
* fixer
* fixing
* libpng friends
* pwd
* ps
* ps
* no fail fast.. for now
* windows
* l
* no powershell
* a
* fk
* fixer
* a
* Revert "fixer"
This reverts commit 76ae72d68686ead01ef914b658c5d8dac5aaee32.
* plzplz
---------
Co-authored-by: Your Name <you@example.com>
* Add statistics entity. Move client stats from Monitor to EventManager
* Fix warning in no_std
* Make rustfmt happy
* Fix more
* Fix with feature tcp_manager on
* Rename more introspection monitor; Remove unnecessary client_stats
* Fix unused import for no_std
* Fix unused import for prometheus_monitor feature on
* Cleanup docs
* make it safe
* aa
* forgot to put it back
* stateful
* comment
* lol
* aa
* aa
* aa
* win
* lol
* lol
* a
* a
* i hate rust
---------
Co-authored-by: Your Name <you@example.com>
* testcase name logic
* implement locking
* implement logic for removing testcase
* minor modifications
* minor modifications to remove_testcase()
* extract generate_name() from trait Input (broken)
* Revert "extract generate_name() from trait Input (broken)"
This reverts commit 9e217be2810a8c33970846cdd380f542975cc05b.
* fix ci errors
* remove CorpusId from generate_name() calls
* toml formatting
* write from file instead of fs
* fmt and clippy
* fix windows clippy
* handle renaming of testcase
* fix failing cmplog test
* overwrite lockfile on remove testcase
* format
* bring back corpus id in generate_name
* missed windows executors hook
* fix failing tests
* some more errors
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Make sure EM and Z remain consistent in InProcessExecutor
* first make them compile
* a
* hah
* fix intel pt
* decouple fuzzer from em
* lol
* 3
* fix tcp
* fix
* fix
* fix
* fixer
* std
* fixer
* plz
* plzplzplz
* plzplzplzplz
* mm
* more
* symbol
* a
* a
* mm
* mmm
* mmmm
* mmmmm
* ff
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
The implementation of [`Lehmer64Rand::next`] performs a mul on `u128`, which
is not checked against overflows. It leads to panic in debug mode.
[`Lehmer64Rand`]: fd6271fa35/libafl_bolts/src/rands/mod.rs (L373-L376)
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Frida updates and FASAN fixes
* fmt
* Fixes
* clippy
* fmt
* Clippy
* Update to frida 0.16.2
* fix windows hooks
* Fix
* Fmt
* windows fix
* Bump frida version to 0.16.3
* Get rid of call to LLVMFuzzerInitialize
* bump version 0.16.5; use find_global_export_by_name
* allow unused_macro_rules
* Don't do stdout_file on windows
* fmt
* Add tmate to debug
* fix windows frida_libpng
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* fix drcov path parsing
* refactoring of drcov tool
* add the possibility to sort addresses in drcov tools
* more aggressive clippy. it now catches more warnings as errors than before
* reduce the number of unfixable warnings displayed.
* Fixed the issue where compilation failed under the simplemgr configuration.
* Fixed the issue where the crash handler would fail during rerun.
* update fmt
---------
Co-authored-by: h1k0 <h1k0naka@outlook.com>
* Completely remove Input as an associated type in multiple traits
* Unify usage of Input as generic instead
* Remove many unused bounds, in particular HasCorpus
* fix multiple generic ordering
* update and fix CONTRIBUTING.md
* update MIGRATION
* use the same generic input type for new / with_max_iterations to make typing easier in most cases.
* Restore libafl_libfuzzer test in CI
(See #2853.)
This commit attempts to improve the [`HasMutatorBytes`] trait to allow it to
be used along with input types that cannot be resizable.
[`HasMutatorBytes`] is split into two traits:
- [`HasMutatorBytes`]: requires `bytes` and `bytes_mut`.
- `HasMutatorResizableBytes`: requires [`HasMutatorBytes`], and also requires
a few other methods for resizing / shrinking the underlying input type.
N.B.: I believe that if merged, this would introduce a breaking change.
[`HasMutatorBytes`]: 198cd5dbc5/libafl/src/inputs/bytes.rs (L26)
* the first step of the last step
* wip
* 99% done
* 99.9% done
* 99.99
* Hello from windows
* aaa
* 99.999
* aa
* 1
* 2
* 3
* 4
* 5
* plz
* plzplzplz
---------
Co-authored-by: Your Name <you@example.com>
* remove UsesInput from libafl_qemu
* Insert I bound instead of ugly Corpus::Input stuff
* Uses a C generic for Commands
* adapt nyx api to new generics
* add qemu linux fuzzers to CI for building.
* more nyx hypercalls implemented, among them:
- panic hypercall
- range filtering hypercall
* fixed some nyx hypercalls behavior.
* added generic read / write to qemu memory
* port linux kernel example to also have nyx API, add better filtering as well.
* make nyx api structs volatile to avoid optimization issues
* Introduce a method create a Vec in place, using a closure.
* use new vec_init function in relevant places.
* removed unused unsafe keywork
* add more allocated memory r/w callbacks
* add more safety notes
* move emulator hooks to separate struct
* update QEMU version
* first step
* wip
* lol
* making progress, slowly
* add
* a bit more
* progress
* 80%done
* 90% done
* done from linux
* Hello from windows
* 1
* 2
* 3
* 4
* 5
* 6
* 7
* Hello from windows
* 9
* 10
* 11
* 11
* 12
* 13
* aa
* fixer
* a
* oops
---------
Co-authored-by: Your Name <you@example.com>
* added change_brk function for correctly handling SYS_brk
* we need to update h.brk with the new brk_val
* map back pages if brk shrunk below the snapshotted value
* fmt and clippy
* use GuestAddr instead of u64
---------
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
* Nyx hypercall API support
* fix linux kernel fuzzer
* hash_me -> hash_64_fast
* fix multiple bug in kernel harness
* do not check libmozjpeg's C files format.
* Qemu config refactoring.
* QEMU error refactoring.
* Single QEMU init function.
* Light refactor of EmulatorModules.
* Qemu is now a parameter to EmulatorModule callbacks and most function hooks.
* EmulatorModules is initialized before QEMU is initialized.
* refactor asan and asanguest modules to avoid custom init of QEMU and use the module interface instead.
* asan fixed size accesses working with generics.
* use pre_syscall_* and post_syscall_* everywhere for consistency.
* adapt qemu_launcher example to fully work with Emulator, since Qemu must now be initialized by Emulator.
* start writing Emulator / EmulatorBuilder / QemuConfig doc.
* fix broken intel pt doc.
* Revert "New year new clippy (#2797)"
This reverts commit deb76555b75ca2ccac83b83a6db1a276c07080b5.
* Mention that program should panic when using InProcessForkExecutor
* Reapply "New year new clippy (#2797)"
This reverts commit 529213ef6334fc18e1898f3cfbd8bed24c312522.
* Added expect error message to TimeFeedback where there used to be an unwrap
* Changed error message handling
* fixed cargo clippy
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Optimize event serialization with pre-allocated buffer
- Added event_buffer field to LlmpEventManager
- Used to_slice instead of to_allocvec
- Pre-allocated buffer size is 4KB
Fixes#1082
* Fallback to to_allocvec in case of event_buffer overflow
Also combined the shared logic between compressed & uncompressed event
firing while keeping the same behavior
* Made the initial event_buffer size to a const
Also removed the unnecessary event_buffer.clear(), since we are already
resizing it
* fixing empty multipart name
* fixing clippy
* New rules for the contributing (#2752)
* Rules
* more
* aa
* Improve Flexibility of DumpToDiskStage (#2753)
* fixing empty multipart name
* fixing clippy
* improve flexibility of DumpToDiskStage
* adding note to MIGRATION.md
* Update bindgen requirement from 0.70.1 to 0.71.1 (#2756)
Updates the requirements on [bindgen](https://github.com/rust-lang/rust-bindgen) to permit the latest version.
- [Release notes](https://github.com/rust-lang/rust-bindgen/releases)
- [Changelog](https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/rust-bindgen/compare/v0.70.1...v0.71.1)
---
updated-dependencies:
- dependency-name: bindgen
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* No Use* from stages (#2745)
* no from stage
* fixer
* doc fix
* how was this working????
* more fixes
* delete more
* rq
* cargo-fuzz
* m
* aa
* Update CONTRIBUTING.md MIGRATION.md (#2762)
* No Uses* from `fuzzer` (#2761)
* go
* fixing stuf
* hello from windows
* more
* lolg
* lolf
* fix
* a
---------
Co-authored-by: Your Name <you@example.com>
* Remove useless cfgs (#2764)
* Link libresolv on all Apple OSs (#2767)
* Somewhat ugly CI fix... (#2768)
* Maybe fix CI
* does this help?
* Very dirty 'fix'
* Add Input Types and Mutators for Numeric Types (#2760)
* fixing empty multipart name
* fixing clippy
* New rules for the contributing (#2752)
* Rules
* more
* aa
* Improve Flexibility of DumpToDiskStage (#2753)
* fixing empty multipart name
* fixing clippy
* improve flexibility of DumpToDiskStage
* adding note to MIGRATION.md
* Introduce WrappingMutator
* introducing mutators for int types
* fixing no_std
* random fixes
* Add hash derivation for WrappingInput
* Revert fixes that broke things
* Derive Default on WrappingInput
* Add unit tests
* Fixes according to code review
* introduce mappable ValueInputs
* remove unnecessary comments
* Elide more lifetimes
* remove dead code
* simplify hashing
* improve docs
* improve randomization
* rename method to align with standard library
* add typedefs for int types for ValueMutRefInput
* rename test
* add safety notice to trait function
* improve randomize performance for i128/u128
* rename macro
* improve comment
* actually check return values in test
* make 128 bit int randomize even more efficient
* shifting signed values
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Add HashMutator
* Fix docs
* Fix docs again
* introducing bloom filter
* fix tests
* Implement evaluate_filtered
* Add macros to libafl_bolts tuples for mapping and merging types (#2788)
* Add macros
* Use the macros for havoc_mutations
* Fix docs
* improve merge_tuple_list_type to accept n items
* libafl_cc: Automatically find llvm_ar path (#2790)
* imemory_ondisk: Don't fail write under any circumstances if locking is disabled (#2791)
* imemory_ondisk: Don't fail write under any circumstances if locking is disabled
* fmt
* inmemory_ondisk: Add a log message on failure
* clippy'
* micro optimization
* Revert changes to global Cargo.toml
* Hide std-dependent dependency behind std feature
* Fix example fuzzer
* Rename constructor for filtered fuzzer
* Reorder generics alphabetically
* Rename HashingMutator, add note to MutationResult about filtered fuzzers
* Improve StdFuzzer according to feedback
* rename hashing mutator
* Fix english in comment
* Cleanup of old PRs that break the CI
* Fix more CI bugs
* Code cleanup
* Remove unnecessary comments
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Sharad Khanna <sharad@mineo333.dev>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: s1341 <s1341@users.noreply.github.com>
* imemory_ondisk: Don't fail write under any circumstances if locking is disabled
* fmt
* inmemory_ondisk: Add a log message on failure
* clippy'
* micro optimization
* Use expect instead of allow, remove unnecessary allows
* Remove more whitelist lint warnings
* tranisitioning more subprojects
* Re-add some necessary allows
* Re-add more required allows
* Some more windows clippy fixes
* Re-add more whitelist items for expect
* More clippy whitelist fun
* Reset changes to generated files
* Reset generated files to correct version
* Move libafl_concolic to expect instead of allow
* Move libafl_frida to expect from allow
* Move libafl_libfuzzer to expect from allow
* Remove more whitelist items for macOS
* Fix unknown clippy allow
* Remove more unnecessary allow statements
* moving fuzzers
* Remove mistakenly added subdirs
* fixing imports
* Remove more unnecessary whitelisted lints
* Fix test for /home/ubuntu/LibAFL/fuzzers/inprocess/libfuzzer_libpng_accounting
* More clippy improvements for libafl_qemu
* fmt
* Some pedantic options
* Fix more stuff
* Remove Little-CMS again
* Add note to static_mut_refs
* Reset the changed testing routine since it is unnecessary
* cleanup warnings in fuzz_anything
* Refactor: Removed MatchNameRef from tuple import
* Used addr_of! macro instead of taking direct reference
* Remove unecessary unsafe block when getting references to SIGNALS array
* Switched from StdMapObserve to ConstMapObserver to create observer
Also updated signals_set to directly modify SIGNALS array
* Format code using fmt_all.sh
* libafl_{cc,derive}: Bump MSRV for `home` crate update
With the upcoming 2024 edition we should also set `resolver = "3"` in
the workspace's Cargo.toml to opt into a new MSRV-aware dependency
resolver. This would fix the breakage we encountered with dependencies
that bump their MSRV in a minor version update.
* libafl_cc: apply clippy suggestion
* fixing empty multipart name
* fixing clippy
* New rules for the contributing (#2752)
* Rules
* more
* aa
* Improve Flexibility of DumpToDiskStage (#2753)
* fixing empty multipart name
* fixing clippy
* improve flexibility of DumpToDiskStage
* adding note to MIGRATION.md
* Introduce WrappingMutator
* introducing mutators for int types
* fixing no_std
* random fixes
* Add hash derivation for WrappingInput
* Revert fixes that broke things
* Derive Default on WrappingInput
* Add unit tests
* Fixes according to code review
* introduce mappable ValueInputs
* remove unnecessary comments
* Elide more lifetimes
* remove dead code
* simplify hashing
* improve docs
* improve randomization
* rename method to align with standard library
* add typedefs for int types for ValueMutRefInput
* rename test
* add safety notice to trait function
* improve randomize performance for i128/u128
* rename macro
* improve comment
* actually check return values in test
* make 128 bit int randomize even more efficient
* shifting signed values
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Fix build target
Create target directory if doesn't exist
* Remove filter on speculatively exec blocks
since also committed blocks can have this flag
* Add current ip_filters getter
* Fix possibile infinite loop in trace decode
* Add comment about target_path
* Make MmapShMem take a AsRef<Path> instead of a byte array
* Make the test actually test if MmapShMem::persist works
* 0.14.1 (#2698)
* 0.14.1
* fixer
* don't build nyx
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Intel PT minor fixes/improvements (#2724)
* waitpid_filtered to ignore SIGWINCH
* Fix warnings unused manifest key: *.version
* Add export_raw feature to libafl_intelpt
* derive Debug for IntelPTHook
* Clippy
* Update target program ELF offsets
* Add comment to KVM pt_mode check
* refactor
* Add intel_pt_export_raw feature in libafl
* map_error instead of unwrap
* borrow checker friendly join_split_trace
and copy trace before deocde to prevent decoding failures
* Set ip_filters (also) with builder
* Move trace to file
* Fix Cargo.toml docs
* Ignore blocks with no instruction
most likely they are filtered out
* Fix assertion in snapshot module for pages that are equal to SNAPSHOT_PAGE_SIZE (#2738)
Co-authored-by: Scott Powell <scott.powell@str.us>
* Remove non_basic const_panic features (#2739)
* Remove non_basic const_panic features
* fmt
* Make FridaInProcessExecutor compatible with TargetBytesConverter, decouple input type from FridaRuntime trait (#2741)
* decouple input type from FridaRuntime trait
* fmt
* fmt2
* remove HasTargetBytes requirement from FridaInProcessExecutor
* fmt
* restore comment
* fix clippy comment error
* adding entry to MIGRATION.md
* only running persist test on unix
* move cfg dependent imports to function
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Marco C. <46560192+Marcondiro@users.noreply.github.com>
Co-authored-by: Scott <55210664+scottmpowell@users.noreply.github.com>
Co-authored-by: Scott Powell <scott.powell@str.us>
Co-authored-by: jejuisland87654 <jejuisland87654@gmail.com>
* waitpid_filtered to ignore SIGWINCH
* Fix warnings unused manifest key: *.version
* Add export_raw feature to libafl_intelpt
* derive Debug for IntelPTHook
* Clippy
* Update target program ELF offsets
* Add comment to KVM pt_mode check
* refactor
* Add intel_pt_export_raw feature in libafl
* map_error instead of unwrap
* borrow checker friendly join_split_trace
and copy trace before deocde to prevent decoding failures
* Set ip_filters (also) with builder
* Move trace to file
* Fix Cargo.toml docs
* Ignore blocks with no instruction
most likely they are filtered out
* launcher now uses client_id instead of core_id
* adding overcommit to an example fuzzer
* Replace addr_of with &raw across the codebase (#2669)
* Replace addr_of with &raw across the codebase
* fix fixes
* more fix
* undo clang fmt?
* oops
* fix?
* allocator fix
* more fix
* more more
* more docs
* more fix
* mas mas mas
* hm
* more
* fix Frida
* needed
* more error
* qemu
* Introduce workspace (again) (#2673)
* Trying to redo workspace deps again after #2672
* unused
* clippy
* Replace addr_of with &raw across the codebase (#2669)
* Replace addr_of with &raw across the codebase
* fix fixes
* more fix
* undo clang fmt?
* oops
* fix?
* allocator fix
* more fix
* more more
* more docs
* more fix
* mas mas mas
* hm
* more
* fix Frida
* needed
* more error
* qemu
* Introduce workspace (again) (#2673)
* Trying to redo workspace deps again after #2672
* unused
* clippy
* fixing formatting issues
* cloning values to make borrow checker happy
* simplifying cfg constraints, removing excessive clippy allows
* printing clang version that is used to find inconsistencies between CI and local formatting
* some fixes according to the CI
* Specifying types
* improved logging for formatter
* more attempts at logging for the CI formatting
* fixing setting LLVM version in formatting in CI
* fixing cippy allows
* renaming launcher's ClientID to ClientDescription
* Lower capped RAND generators (#2671)
* Lower capped rand generators
* Updated all references to RAND generators
* Formatting updates
* New RAND bytes generator constructor
* Revert "Updated all references to RAND generators"
This reverts commit 9daad894b25ec3867daf93c4fe67c03abec1d8c6.
* Revert "Formatting updates"
This reverts commit ff2a61a366c48b3f313878f62409e51b1e1ed663.
* cargo nightly format
* Added must_use to with_min_size
* fix error '#' is not followed by a macro parameter (#2678)
* Use version.workspace (#2682)
* LibAFL_QEMU: Don't return a generic Address from Register reads (#2681)
* LibAFL_QEMU: Make ReadReg always return GuestReg type
* Don't return a generic address
* fix fuzzers
* fix mips
* Add DrCovReader to read DrCov files and DrCov dumper and merge utils (#2680)
* Add DrCov Reader
* Removed libafl_jumper deps
* Fix DrCovWriter, add dump_drcov_addrs
* Taplo
* Move frida from usize to u64
* DrCov usize=>u64
* Better error print
* More u64
* ?
* debug
* clippy
* clippy
* Add Merge option to DrCovReader
* Add drcov_merge tool
* Move folder around
* DrCov
* More assert
* fmt
* Move around
* Fix print
* Add option to read multiple files/full folders
* Fix build_all_fuzzers.sh for local runs (#2686)
* Add Intel PT tracing support (#2471)
* WIP: IntelPT qemu systemmode
* use perf-event-open-sys instead of bindgen
* intelPT Add enable and disable tracing, add test
* Use static_assertions crate
* Fix volatiles, finish test
* Add Intel PT availability check
* Use LibAFL errors in Result
* Improve filtering
* Add KVM pt_mode check
* move static_assertions use
* Check for perf_event_open support
* Add (empty) IntelPT module
* Add IntelPTModule POC
* partial ideas to implement intel pt
* forgot smth
* trace decoding draft
* add libipt decoder
* use cpuid instead of reading /proc/cpuinfo
* investigating nondeterministic behaviour
* intel_pt module add thread creation hook
* Fully identify deps versions
Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates
* Move mem image to module, output to file for debug
* fixup! Use static_assertions crate
* Exclude host kernel from traces
* Bump libipt-rs
* Callback to get memory as an alterantive to image
* WIP Add bootloader fuzzer example
* Split availability check: add availability_with_qemu
* Move IntelPT to observer
* Improve test docs
* Clippy happy now
* Taplo happy now
* Add IntelPTObserver boilerplate
* Hook instead of Observer
* Clippy & Taplo
* Add psb_freq setting
* Extremely bad and dirty babyfuzzer stealing
* Use thread local cell instead of mutex
* Try a trace diff based naive feedback
* fix perf aux buffer wrap handling
* Use f64 for feedback score
* Fix clippy for cargo test
* Add config format tests
* WIP intelpt babyfuzzer with fork
* Fix not wrapped tail offset in split buffer
* Baby PT with raw traces diff working
* Cache nr_filters
* Use Lazy_lock for perf_type
* Add baby_fuzzer_intel_pt
* restore baby fuzzer
* baby_fuzzer with block decoder
* instruction decoder instead of block
* Fix after upstream merge
* OwnedRefMut instead of Cow
* Read mem directly instead of going through files
* Fix cache lifetime and tail update
* clippy
* Taplo
* Compile caps only on linux
* clippy
* Fail compilation on unsupported OSes
* Add baby_fuzzer_intel_pt to CI
* Cleanup
* Move intel pt + linux check
* fix baby pt
* rollback forkexecutor
* Remove unused dep
* Cleanup
* Lints
* Compute an edge id instead of using only block ip
* Binary only intelPT POC
* put linux specific code behind target_os=linux
* Clippy & Taplo
* fix CI
* Disable relocation
* No unwrap in decode
* No expect in decode
* Better logging, smaller aux buffer
* add IntelPTBuilder
* some lints
* Add exclude_hv config
* Per CPU tracing and inheritance
* Parametrize buffer size
* Try not to break commandExecutor API pt.1
* Try not to break commandExecutor API pt.2
* Try not to break commandExecutor API pt.3
* fix baby PT
* Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620)
* support (unsafe) on_crash / on_timeout callbacks for modules
* use libc types in bindgen
* Move common code to bolts
* Cleanup
* Revert changes to backtrace_baby_fuzzers/command_executor
* Move intel_pt in one file
* Use workspace deps
* add nr_addr_filter fallback
* Cleaning
* Improve decode
* Clippy
* Improve errors and docs
* Impl from<PtError> for libafl::Error
* Merge hooks
* Docs
* Clean command executor
* fix baby PT
* fix baby PT warnings
* decoder fills the map with no vec alloc
* WIP command executor intel PT
* filter_map() instead of filter().map()
* fix docs
* fix windows?
* Baby lints
* Small cleanings
* Use personality to disable ASLR at runtime
* Fix nix dep
* Use prc-maps in babyfuzzer
* working ET_DYN elf
* Cleanup Cargo.toml
* Clean command executor
* introduce PtraceCommandConfigurator
* Fix clippy & taplo
* input via stdin
* libipt as workspace dep
* Check kernel version
* support Arg input location
* Reorder stuff
* File input
* timeout support for PtraceExec
* Lints
* Move out method not needing self form IntelPT
* unimplemented
* Lints
* Move intel_pt_baby_fuzzer
* Move intel_pt_command_executor
* Document the need for smp_rmb
* Better comment
* Readme and Makefile.toml instead of build.rs
* Move out from libafl_bolts to libafl_intelpt
* Fix hooks
* (Almost) fix intel_pt command exec
* fix intel_pt command exec debug
* Fix baby_fuzzer
* &raw over addr_of!
* cfg(target_os = "linux")
* bolts Cargo.toml leftover
* minimum wage README.md
* extract join_split_trace from decode
* extract decode_block from decode
* add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0
* More generic hook
* fix windows
* Update CI, fmt
* No bitbybit
* Fix docker?
* Fix Apple silicon?
* Use old libipt from crates.io
---------
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* libafl-fuzz: introduce nyx_mode (#2503)
* add nyx_mode
* fix frida ci?
* damn clippy
* clippy
* LibAFL: Remove `tui_monitor` from default features (#2685)
* No Usermode default
* no tui
* gg
* try fix CI
* fmt
---------
Co-authored-by: Dominik Maier <dmnk@google.com>
* Actually make ConstMapObserver work, introduce `nonnull_raw_mut` macro (#2687)
* Actually make ConstMapObserver work
* fixes
* does that work?
* mas
* Feature: libafl-fuzzfuzzbench (#2689)
* fuzzbench
* clippy
* fmt
* fix unicorn CI?
* Move bitfields to bitbybit (#2688)
* move to bitbybit
* Restore bitbybit dependent code
* Clippy
* Fix NautilusContext::from_file for python files (#2690)
* Bump to 0.14.0 (#2692)
* Fix versions in libafl and libafl_intelpt for crates.io (#2693)
* Fix versions in libafl and libafl_intelpt for crates
* Add libafl_intelpt to publish
* StdMOptMutator:🆕 remove unused type parameter (#2695)
`I` is unused in `::new` and thus requires callers to explicitly specify
any type as it can't be determined by type inference.
Clippy's `extra_unused_type_parameters` should pick this up, but is
tuned a bit too conservative in order to avoid false positives AFAICT.
* Move test_harness from source directory to OUT_DIR (#2694)
* remove test_harness from source directory
* fmt
* Add package.metadata.docs.rs for libafl_intelpt (#2696)
* libafl-fuzz: fix cmplog running on inputs more than once (#2697)
* libafl-fuzz: fix cmplog running on inputs more than once
* fmt
* fix afl++ cmplog header
* update to latest afl stable commit
* Libafl workspace internal deps in workspace Cargo.toml (#2691)
* Add internal deps to workspace
* libafl: use workspace internal deps
* libafl_bolts: use workspace internal deps
* 0.14.0
* use workspace internal deps
* Fix tui monitor for example fuzzers (#2699)
* Fix tui monitor for example fuzzers
* New clippy lint
* fix
* Update pyo3-build-config requirement from 0.22.3 to 0.23.1 (#2701)
Updates the requirements on [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.22.3...v0.23.1)
---
updated-dependencies:
- dependency-name: pyo3-build-config
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* bolts: fix build for tiers 3 platforms. (#2700)
cater to platforms knowingly support this feature instead.
* Pre init module hooks (#2704)
* differenciate pre qemu init and post qemu init hooks
* api breakage: Emulator::new_with_qemu is not public anymore.
* Fix edge module generators (#2702)
* fix generators
* fix metadata removal for ExecutionCountRestartHelper (#2705)
* Ignore pyo3 update (#2709)
* libafl-fuzz: feature-flag nyx mode (#2712)
* Bump ctor dependency to make nightly compile again (#2713)
* Batched timeout doc (#2716)
* timeout doc
* clp
* FMT
* More batched timeout doc (#2717)
* timeout doc
* clp
* FMT
* more
* fixing an overexited cast
* renaming variables
* removing unnecessary brackets
* fixing imports
* fixing imports
* renaming more variables
* even more variable renaming
* removing duplicate clap short options
* reverting mistaken variable renaming
* comparing the actual cores instead of an enumeration index
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Subhojeet Mukherjee, PhD <57270300+CowBoy4mH3LL@users.noreply.github.com>
Co-authored-by: jejuisland87654 <jejuisland87654@gmail.com>
Co-authored-by: Marco C. <46560192+Marcondiro@users.noreply.github.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Aarnav <aarnav@srlabs.de>
Co-authored-by: Dominik Maier <dmnk@google.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: Mrmaxmeier <3913977+Mrmaxmeier@users.noreply.github.com>
Co-authored-by: Sharad Khanna <sharad@mineo333.dev>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: Henry Chu <henrytech@outlook.com>
* Make exit status interpretable by CommandConfigurator
* Fix import issues
* Fix default implementation for non-unix environment
* Make docs only available on unix if the entry is only for unix
* Revert "Fix default implementation for non-unix environment"
This reverts commit 5457f6f7376c2a3a4d4c8459de46d6b54bb0d44f.
* Fix the invalid link in the example
* update pyo3 to latest version
* add python bindings to workspace
* make pyo3 stuff dependent of workspace again
* adapt implementation for the newest version of pyo3
`I` is unused in `::new` and thus requires callers to explicitly specify
any type as it can't be determined by type inference.
Clippy's `extra_unused_type_parameters` should pick this up, but is
tuned a bit too conservative in order to avoid false positives AFAICT.
* WIP: IntelPT qemu systemmode
* use perf-event-open-sys instead of bindgen
* intelPT Add enable and disable tracing, add test
* Use static_assertions crate
* Fix volatiles, finish test
* Add Intel PT availability check
* Use LibAFL errors in Result
* Improve filtering
* Add KVM pt_mode check
* move static_assertions use
* Check for perf_event_open support
* Add (empty) IntelPT module
* Add IntelPTModule POC
* partial ideas to implement intel pt
* forgot smth
* trace decoding draft
* add libipt decoder
* use cpuid instead of reading /proc/cpuinfo
* investigating nondeterministic behaviour
* intel_pt module add thread creation hook
* Fully identify deps versions
Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates
* Move mem image to module, output to file for debug
* fixup! Use static_assertions crate
* Exclude host kernel from traces
* Bump libipt-rs
* Callback to get memory as an alterantive to image
* WIP Add bootloader fuzzer example
* Split availability check: add availability_with_qemu
* Move IntelPT to observer
* Improve test docs
* Clippy happy now
* Taplo happy now
* Add IntelPTObserver boilerplate
* Hook instead of Observer
* Clippy & Taplo
* Add psb_freq setting
* Extremely bad and dirty babyfuzzer stealing
* Use thread local cell instead of mutex
* Try a trace diff based naive feedback
* fix perf aux buffer wrap handling
* Use f64 for feedback score
* Fix clippy for cargo test
* Add config format tests
* WIP intelpt babyfuzzer with fork
* Fix not wrapped tail offset in split buffer
* Baby PT with raw traces diff working
* Cache nr_filters
* Use Lazy_lock for perf_type
* Add baby_fuzzer_intel_pt
* restore baby fuzzer
* baby_fuzzer with block decoder
* instruction decoder instead of block
* Fix after upstream merge
* OwnedRefMut instead of Cow
* Read mem directly instead of going through files
* Fix cache lifetime and tail update
* clippy
* Taplo
* Compile caps only on linux
* clippy
* Fail compilation on unsupported OSes
* Add baby_fuzzer_intel_pt to CI
* Cleanup
* Move intel pt + linux check
* fix baby pt
* rollback forkexecutor
* Remove unused dep
* Cleanup
* Lints
* Compute an edge id instead of using only block ip
* Binary only intelPT POC
* put linux specific code behind target_os=linux
* Clippy & Taplo
* fix CI
* Disable relocation
* No unwrap in decode
* No expect in decode
* Better logging, smaller aux buffer
* add IntelPTBuilder
* some lints
* Add exclude_hv config
* Per CPU tracing and inheritance
* Parametrize buffer size
* Try not to break commandExecutor API pt.1
* Try not to break commandExecutor API pt.2
* Try not to break commandExecutor API pt.3
* fix baby PT
* Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620)
* support (unsafe) on_crash / on_timeout callbacks for modules
* use libc types in bindgen
* Move common code to bolts
* Cleanup
* Revert changes to backtrace_baby_fuzzers/command_executor
* Move intel_pt in one file
* Use workspace deps
* add nr_addr_filter fallback
* Cleaning
* Improve decode
* Clippy
* Improve errors and docs
* Impl from<PtError> for libafl::Error
* Merge hooks
* Docs
* Clean command executor
* fix baby PT
* fix baby PT warnings
* decoder fills the map with no vec alloc
* WIP command executor intel PT
* filter_map() instead of filter().map()
* fix docs
* fix windows?
* Baby lints
* Small cleanings
* Use personality to disable ASLR at runtime
* Fix nix dep
* Use prc-maps in babyfuzzer
* working ET_DYN elf
* Cleanup Cargo.toml
* Clean command executor
* introduce PtraceCommandConfigurator
* Fix clippy & taplo
* input via stdin
* libipt as workspace dep
* Check kernel version
* support Arg input location
* Reorder stuff
* File input
* timeout support for PtraceExec
* Lints
* Move out method not needing self form IntelPT
* unimplemented
* Lints
* Move intel_pt_baby_fuzzer
* Move intel_pt_command_executor
* Document the need for smp_rmb
* Better comment
* Readme and Makefile.toml instead of build.rs
* Move out from libafl_bolts to libafl_intelpt
* Fix hooks
* (Almost) fix intel_pt command exec
* fix intel_pt command exec debug
* Fix baby_fuzzer
* &raw over addr_of!
* cfg(target_os = "linux")
* bolts Cargo.toml leftover
* minimum wage README.md
* extract join_split_trace from decode
* extract decode_block from decode
* add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0
* More generic hook
* fix windows
* Update CI, fmt
* No bitbybit
* Fix docker?
* Fix Apple silicon?
* Use old libipt from crates.io
---------
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* introducing Launcher::overcommit
* removing unnecessary cfg restrictions and clippy allows
* improving warning for wrong clang-format version
* installing black in the format CI
* Enforcing python formatting in CI
* extending formatting using black on all python files
* printing diff on black failure
* preferring python's black over system black
* moving to LLVM 19 for formatting
* Replace addr_of with &raw across the codebase
* fix fixes
* more fix
* undo clang fmt?
* oops
* fix?
* allocator fix
* more fix
* more more
* more docs
* more fix
* mas mas mas
* hm
* more
* fix Frida
* needed
* more error
* qemu
* clean clippy warnings from fuzzers/binary_only/*
* handle unused Results in fuzzers/binary_only/*
* format fuzzers/binary_only/qemu_cmin
* use unchecked memory write in qemu fuzzer examples
* create file_null in fuzzbench_fork_qemu
* compile-time edge module compilation trick
* clippy
* possible since rust 1.79
* split edge module in submodules
* Update frida to 0.14.0 (#2596)
* update frida crate to the latest version
* adapt libafl_frida to the latest version of frida
* tracers and generators private modules
* do not use star export.
* same for drcov
* forgot a file...
* first draft of generic-based edge module for ConstantLengthMapObserver.
* integration of OwnedSizedSlice.
replaced OwnedSlice in ConstMapObserver by the new OwnedSizedSlice.
* fix serde stuff
* no std
* import
* fixed qemu_cmin with new constant map abstraction.
* fix const map
* fix clippy from another pr...
* fix non-null usage
* fix ci?
* new feature stuff
* fixes
* minor fixes
* fmt
* non null
* im stupid
* fmt
* fix fuzzer
* fix fuzzers
* sized slice
* fuzzer fixes
* ptr::NonNull -> NonNull
* shorter trait length
* fmt
* libafl_qemu: Add RISCV support
Adds the following targets (as features):
- riscv32
- riscv64
Added `RISCVCPU` and `CPURISCVState` to the bindings allow list.
Added riscv.rs to the arch module, with all necessary functions and
registers implemented and mapped.
The registers are the same as the ones found in qemus gdbstub xml found
after a build.
Additionally we added all syscall numbers for riscv 64 bit (already
supported by the `syscall_numbers` crate) and also added the missing
ones for riscv 32 bit. We compared both lists and their differences /
equalities with a simple python script and generated a list of the
missing ones, to be complete.
We might PR those to the `syscall_numbers` crate later on.
---------
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
* feat(libafl_core): make executors and feedbacks more cleanly usable outside of LibAFLs Fuzzer loop
* cargo +nightly fmt
* updated type constraints
* reformatted and final type constraint fixes
* made unicode extraction stage useful separately
* fix libafl_cc error message
* fix state type constraint to be constrained on the method
* removed unnecessary observer constraint
* renamed unused variables
* fix unnecessary error wrapping in helper functions
* converted unicode conversion stage into associated function and fixed nautilus changes
* more update
* Remove extra I
* more fmt
* bounds?
* less bounds
* more less bounds
* different trait bounds again
* more less generics
* fix unicode
* fix list
* remove unneeded bound
---------
Co-authored-by: Lukas Dresel <Lukas-Dresel@users.noreply.github.com>
Co-authored-by: Toka <tokazerkje@outlook.com>
* add python grammar loader for Nautilus
* fmt
* fmt toml
* add python to macos CI deps
* install python
* fmt
* ci
* clippy
* fix workflow
* fmt
* fix baby nautilus
* fix nautilus sync
* fmt
* fmt
* clippy
* typo
* fix miri
* remove pyo3 from workspace to packages which need it and make it optional
* go back to AsRef<Path> for nautilus grammar loading
* replace hardcoded python flags for macos build
* typo
* taplo fmt
* revert formatting of libafl_qemu_arch
* ci
* typo
* remove expects in NautilusContext::from_file and make them Results
* remove not(miri) clause in test
* try and fix python build fir ios and android
* again
* android
* tmate
* fix android build
* document load_python_grammar
* log if python or json when loading nautilus grammar
* make nautilus optional
* add nautilus as feature to forkserver_simple_nautilus
* Remove emulation_mode env variable and custom cfg
* Using only the feature flag simplifies things a bit and allow the usage of optional dependencies
* Do not use --all-features on libafl_qemu
* Add missing target_os = "linux"
* nonzero_lit
* nonzero
* std instead of core
* a
* l
* test
* import
* api
* api
* aaaaa
* apiapi
* api
* api
* api
* mm
* api
* non zero
* FMT
* pls
* nnnaaasdfadsfafdsa
* pls
* MM
* fix
* a
* sat add
* aa
* mistake
* unreachable
* no generic
* api change
* a
* centralize clippy definition
* fmt
* add update bindings script
* add a checked and unchecked version of memory read to qemu stuff
also, a lot of clippy thing
* update binding position
* rm old script, new one is a bit better
* update doc
* macos clippy
* adapt fuzzers
* windows clippy
* fix fuzzer
* windows clippy
* remove old allowed clippy
* remove some allowed clippy
* use default features for serde_json in gramatron
* better error handler for failed rw to memory
* Fix read_st_size in forkserver
* fixed read_st, read_st_size, and write_ctl
with read_exact and write_all
* Cleanup redundant return values in forkserver pipe I/O
* Forkserver: avoid using read_exact/write_all for reading/writing integers
* Forkserver: avoid initializing the vec twice when reading a chunk of bytes
* Fix code formatting
* change fuzzbench_qemu
* real test
* fix qemu crash hook
* update bindings
* fix fork executor, reduce trait bound overhead
* make EdgeModule depend on observer to get ptrs.
* do not make EdgeCoverageModule::new public
* map observer as builder call
* adapt examples with new edge coverage module builder.
* TMP: everyone is a variable length map observer
* reuse profile path script
* fix absolute paths
* remove some dependencies to make pipeline faster
* compile-time builder initialization check
---------
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
* versioning unification: use x.y.z format everywhere
* do not compile low-level logs (< info level) by default in fuzzers
* update dependencies to the latest versions
* add members to workspace.
* use workspace for common dependencies
* add vscode native support
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
* Rand below should take a NonZero parameter
* More
* more
* More
* fix build
* bit of clippy
* more clippy
* more clippy
* More clippy
* More more
* more nonzero
* fix multipart
* Cleanup, more unsafe
* fix
* fix unicode
* clippy, fmt
* more
* More safer and more better
* MaxStackPow
* fix merge fails
* make random_slize_size faster
* fix
* more
* fix
* add from addison's change
* fixing
* one left
* adding
* 444444
* okoko
* no prelude
* i hate rust
* stuff
* no std/win
* getting closer
* a
* a
* aa
* a
* a
* libafl_fuzz
* toml
* tutorial
* libafl_libfuzzer
* Mark unsafe functions unsafe, as Ferris inteded
* More
* more safety?
* more fix
* actually safe
* More cleanup
* More fix
* more unsafe
* fix imports
* more unsafe
* fixes
* bring back the memories
* Update README.md
Re-structure the first page to be a bit more appealing
- Regrouping getting started and fuzzer example in one sub-section
- Separate installation guide in a different sub-section
- Moving 'Why LibAFL'' further in the page
- Removing spacing/newline to group info together
* Update README.md
- Fixing typo
- Improving layout
* Update README.md
Cleaner readme
- overview and why libafl merged
- resources and quick started cleaned
* Update README.md
- Build instruction before getting started
* Update README.md
* Update README.md
- Separate Why LibAFL and Overview section
* Update README.md
* Update README.md
* Update README.md
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Move fuzzers around some more
* back to baby
* this was missing..
* shuffeling shuffeling
* shuffeling
* md
* cleanup
* oops
* Move foldername to underscore
* more doc
* Add links
* cleanup
* Move fuzzers around some more
* back to baby
* this was missing..
* shuffeling shuffeling
* shuffeling
* md
* cleanup
* oops
* Move foldername to underscore
* more doc
* wip
* this shit is crazy
* from windows
* fmt
* fuck
* fmt
* aa
* reduce generics for generator
* do stuff for mutators, too
* aa
* a
* fmt
* idk
* getting things compiling
* fix merge
* idk
* fuzzers
* nire
* more
* a
* frida win
* cmin
* mm
* qemu?
* del ubuntu dep
---------
Co-authored-by: Your Name <you@example.com>
* linux kernel (x509_cert) and process fuzzing example
* rework filters
* update to latest qemu
* working for process and kernel fuzzing
* new i2s mutator for binary only fuzzers
* refactoring modules with new filtering interface
* add state as parameter of harness
* hide unused global in usermode
* Script for stub bindings generation
* do not try to check whether it is worth generating the bindings, always
generate when the env variable is on.
* add taplo to fmt_all.sh
* Moved fuzzers (again) in a target-centric way.
* fix rust 2024 warnings.
* new libafl_qemu harness structure.
* rename qemu_systemmode into qemu_baremetal
* fix qemu baremetal makefile
* fix formatter
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
* Add some knowledge of const parameters in cmplogs
* Careful with the CmpLog unions and clean macros
* Fix for macros?
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* introducing MappingMutator
* extending mapping mutators
* adding example fuzzer
* making crossover mutators more flexible.
* moving example fuzzer
* fixing dependency paths
* formatting
* fixing no std error
* fixing broken docs link
* fixing import paths
* fixing imports
* more format fixing
* adding new example fuzzer to CI
* fixing further imports
* fixing formatting
* formatting fixes
* improving docs for the example fuzzer
* adding documentation and tests to mapping mutators
* make extraction function for mapped crossover mutators more general
* adding MutVecFunctionMappingMutator
* Introducing WrapsReference
* code cleanup for mapping mutators
* adding tests and docs to mapping mutators
* reformatting comments
* fixing merging of mutators in example fuzzer
* formatting
* formatting v2
* cleanup according to PR comments
* adding type constraint to MappedInput helper functions to remove the need to specify types
* matching functions passed to mapped_havoc_mutations
* removing unnecessary constraints
* mapping mutators now contain the name of their inner mutator
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* feat: revert now unecessary workaround, use rust-lang.rust-analyzer
* fix: specify llvm version and symlink llvm. Thanks @tokatoka
* fix: pass the llvm version to createAliases.sh
* fix: shell script lints
* fix: shell script lints
* feat: use cargo binstall to make the container build faster
- Only display 1 digit after . for value above 1k
- Only display 2 digits after . for value above 1m
3.254k exec/sec becomes => 3.2k exec/sec
3.254M exec/sec becomes => 3.25M exec/sec
* Better documentation headers (clippy)
* more doc
* more fixes
* Even more
* more
* even more
* concrete
* fmt
* even more more
* tiny typo
* more
* more
* More
* more
* more docs?
* more docs
* bolts: Support dump_registers on Windows/x86
* bolts: Support dump_registers on Windows/aarch64
* bolts: Fix sig_ign() on Windows/x86
* bolts: Fix format of dump_registers under Windows
* bolts: Add test for dump_register under Windows
* the rustup official command to update is 'rustup update' not 'rustup upgrade', although to catch mistakes, 'rustup upgrade' also fallback to update
* fix references to baby_fuzzer and baby_fuzzer new location
* modernize a bit the manifests and code snippets for the baby_fuzzer to make the experience more consistent with the current rust compiler
* fix another reference path for baby fuzzers
* fix frida fuzzers locations in the book
* fix the concolic hybrid fuzzers location in the book
* fix baby_no_std location in the book
* fix nyx fuzzers location in the book
* Revert "modernize a bit the manifests and code snippets for the baby_fuzzer to make the experience more consistent with the current rust compiler"
This reverts commit c58bfbfffd06a3afa8338043dde6ed59b03d8178.
* add markdown link check in ci
* fix broken links when possible.
configuration file for md checker.
* accept code 200...
* more fixes
* more fixes
* more fixes
* Add a custom typed builder for Emulator
* Unify qemu_init for usermode and systemmode
* Remove env from qemu init args (it is unused in QEMU in practice)
* expose thread hooks to systemmode
* rename qemu_config to config
* Replace ExitHandler by EmulatorDriver
* Reorder generics alphabetically for Qemu{,Fork}Executor
* Moved snapshot manager to Emulator to continue centralizing mains objects in the same structure
* Reimplementation of CommandManager working with enums instead of tables
* Macro has been adapted to do this work automatically
* Moved snapshot stuff to dedicated module
* Removed many Rc<RefCell<...>>, now useless with the removal of vtables
* Builder given by Emulator via `Emulator::builder`. Reduced trait bound overhead
* Remove stateless qemu executor
* All harnesses take a reference to an emulator as parameter now
* harness takes an emulator as first parameter, and input as second parameter (opposite of previous definition)
* bump libafl qemu dependencies to the latest version
* libafl-fuzz: simplify Makefile.toml
* Re-introduce support for old AFL++ forkserver
* clippy
* libafl-fuzz: add support for QEMU mode
* libafl-fuzz: simplify Makefile
* Update LibAFL QEMU to the latest version (V9.0.2 update, important bug fixes, ... - check the dedicated repo for more info)
* fix bug in hook execution, causing first execution hooks to be run multiple times.
* 0.13.2 (for real this time)
* More 0.13.2
* Make sure this never bites us
* Update ahash
* Fix CI after rust 1.80
* Does 18 work?
* Trying to fix this?
* lol regex
* noble
* works?
* fix llvm-config
* fmt
* Windows?
* more fix?
* more fix
* remove pip
* Add find_python3_version support to libafl_cc
* llvm-config
* more curl
* windows has no apt lol
* more
* fix cmplog implementation
only set testcase filepath if filepath is none
* libafl-fuzz: fix minor CI
* add missing fields to AFLppCmpLogOperands
* libafl-fuzz: pin CI AFL version to a commit
fix extended_cmplog_instrumentation
* libafl-fuzz: fix CI
* this should not panic
* aaa
* libafl-fuzz: fix cmplog CI
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
* CI: update to checkout@v4 action
Not sure why they decided to bump a major version. It seems like this is
unnecessary churn: https://github.com/actions/checkout/pull/1436#issuecomment-1737711301
* CI: replace archived `actions-rs/toolchain`
It's been unmaintained for four years and has been throwing warnings in
CI:
> The `set-output` command is deprecated and will be disabled soon.
* CI: Bump `cargo-install` action.
This should fix the remaining warnings in CI?
* CI: mdbook-linkcheck: workaround for build error
* libafl-fuzz: add cmplog to CI
libafl-fuzz: add option to specify custom rng sed
libafl-fuzz: add help messages to CLI, add file extension support
libafl-fuzz: adhere to AFL++ cmplog bin path format
libafl-fuzz: avoid races when writing to fuzzer_stats
libafl-fuzz: add time tracking for CalibrationStage, MutationalStage and SyncFromDiskStage
* libafl-fuzz: fix libafl paths
* libafl-fuzz: remove redundant cmplog check
* libafl-fuzz: ingore UnstableMapEntries when using queue scheduler in afl_stats.rs
libafl-fuzz: track max_depth for QueueScheduler
libafl-fuzz: fix custom input file
* libafl-fuzz: fix Makefile
* clippy
use of LLVM 19 by default
LLVM version should be much easier to switch from now on
a lot of code repetition has been deleted
removed llvm-related github action
fix format check
other small things
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
* push
* upd
* add last found time
* add common as prerequisite
* clp
* aa
* more clp
* fix how to get corpus id
* pruning
* aa
* no std
* fix
* working?
* push
* Helper is now called Module.
* Emulator now contains hooks state.
* Emulator is managed by QemuExecutor.
* QEMU hooks have been completely refactored on the rust side.
* Generics cleanup.
* Introduce libafl-fuzz
* fix corpus file path
* simplify SeedFeedback
* fix incorrect comment
* add clap::ValueEnum to PowerSchedule as an optional derive if clap is enabled
* UnixShMemProvider replaced with StdShMemProvider for libafl-fuzz
* remove io_error_more feature constraint
* libafl-fuzz: make Ok(()) unreachable in CentralizedLauncher
* libafl-fuzz: make harness_input_stdin to harness_input_type with &'static
* libafl-fuzz: move each feedback to it's own file
* make run_fuzzer_with_stage into a function.
use CachedOnDiskCorpus instead of OnDiskCorpus for Corpora
remove utils file
* remove unecessary clone
* libafl-fuzz: cleanup AFLStatsStage
* move peak_rss_mb to libafl_bolts
* parse envs by hand
* add sensible defaults for map size and broker port
* fix test.sh and corpus_id padding
* add Makefile.toml
* libafl-fuzz update test suite
* libafl-fuzz: clippy
* rename peak_rss_mb to peak_rss_mb_children
* remove minor version constraint for clap
* libafl-fuzz: fix ELF check and instrumentation check in check_binary
* libafl-fuzz: improve Makefile.toml
* simplify fuzzer and cleanup typos
* libafl-fuzz: load corpus entries in a multicore fashion
* libafl-fuzz: create output dir if not exists (non-racey)
* libafl-fuzz: add sequential scheduling support
libafl-fuzz: add cmplog options
libafl-fuzz: add test-cmplog.c to CI
* rename peak_rss_mb_children to peak_rss_mb_child_processes
* fix race condition in SyncFromDiskStage, add interval based checking and support for multiple directories.
libafl-fuzz: add support for syncing with foreign fuzzers
* update README
* implement AflScheduler for QueueScheduler.
Add queue_cycles field to
AflScheduler
* libafl-fuzz: remove dependecy on SchedulerMetadata for AflStatsStage
* clippy
* remove queue_cycles from AflScheduler into int's own trait.
libafl-fuzz: simplify map observer tracking
* clippy
* libafl-fuzz: disable cmplog check in CI for now
* add missing constraints for libafl_qemu executor
* clippy
* libafl-fuzz: improve Makefile
libafl-fuzz: clippy
* libafl-fuzz: misc
* misc typos, beautify
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* aa
* why??????????????
* ppp
* aa
* aa
* abcde
* fixer
* ppp
* aa
* fix from windows
* sugar
* ff
* ??
* a
* to log::error
* exclude
* exclude libafl_qemu clippy on windows
* pp
* aa
---------
Co-authored-by: Your Name <you@example.com>
* Introduce TuiMonitor builder
* Some random docs
* More documentation for monitors
* fixed critical whitespace
* Rename all-caps TOML and JSON to Toml and Json in monitors
* actually rename
* more
* doc(libafl_nyx): More detailed README
* doc(libafl): Documentation about mutators
* doc(libafl): fix Reference to MOpt
* doc(libafl): Improved note about Mopt
* doc: More documentation for different mutators
The length constraint on the bottom areas was set to 0 and thus those two blocks were invisible.
We adjusted those to the correct values to show all lines & also removed the unnecessary extra chunk layout variable.
* doc: fixed instruction in CONTRIBUTING.md
* Added note for windows users
* fix: Added corpus for dynamic_analysis fuzzer
* fix for calling the clippy script on windows
* Fixed path oversight in previous commit
* add HasStopNext to State which exposes an API to stop the fuzzer. Stops the fuzzer in fuzz_loop or
fuzz_loop_for when set to true
* fix import
* rename HasStopNext to HasShouldStopFuzzing and stop_next to should_stop_fuzzing
* added HasShouldStopFuzzing trait constraint for libafl_libfuzzer_runtime fuzzer
* rename HasShouldStopFuzzing to Stoppable and add it as a type constraint in libafl_libfuzzer report.rs
* rename should_stop_fuzzing -> should_stop
* introduce Event::Stop
* fix prelude import
* Call send_exiting when processing Event::Stop in restartable managers
* fix clippy
* introduce on_shutdown function in EventProcessor, a function to exit
without saving state gracefully. In contrast with on_restart.
* call manager.on_shutdown when stopping in fuzz_loop due to state.should_stop
* Add missing on_shutdown implementations
Check after every stage in Stages::perform_all if should exit and do so.
* remove specialization
* fix doc
* introduce EventProcessor constraint in libafl_libfuzzer_runtime
run clippy in libafl_libfuzzer_runtime
* fix CentralizedEventManager's on_shutdown not calling inner.on_shutdown
* fix bugs in CentralizedLauncher that wouldn't allow children to terminate properly
* don't call send_exiting when processing Event::Stop since it will be called when calling on_shutdown anyways
* clippy
* add set_exit_after so broker does not need to inner_mut to set exit_cleanly_after
* return Cow<str> from Event::name_detailed instead of a String
* fix missing import in libafl_libfuzzer_runtime
* add initate_stop and reset_stop to Stoppable trait to superceed should_stop_mut
* clippy
* typo
* rename initate_stop to request_stop, should_stop to stop_requested and reset_stop to discard_stop_request
* fix missing import
* windows clippy fix
* fix broker typo
* doesn't work poc
* Works
* make this work with or without feature
* start time
* Fix fixes
* Fix more build
* fix build
* reset changes in fuzzbench fuzzer
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
* All Ids should be id not idx
* More
* fix
* win?
* win?
* win?
* fix
* more fix
* desyscall?
* more
* fmt
* std
* make id less id-y
* fmt
* fix
* cleanup
* fixes all around
* fix
* Fix the broken stuff from refactoring
* remove unused
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Fix publish errors
* publish.sh
* remove include_str readme in libafl_qemu
* execute libafl_qemu_build::maybe_generate_stub_bindings() only if in cargo doc
* add support for cycling PowerSchedule on completion of a queue cycle in WeightedScheduler
* improve doc
* make fn cycle_schedule private
* rename cycle_schedules to cycling_scheduler
* Fix docs on crates.io for LibAFL_Frida, introduce auto-download feature
* remove it more
* more testing
* more features
* more CI
* CI?
* CI?
* More fix?
* nicer
* More fix?
* test?
* more try?
* why?
* more more
- features check was inverted
- print helpful message when `llvm-nm` wasn't found, which happens to be
the case on _stable_
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* change Stability calculation formla to AFL++'s
* clippy
* use MapFeedbackMetadata instead of recalculating filled entries in map
* calculate filled entries if MapFeedbackMetadata is not available
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* generic inner manager for centralized, with builder closures.
* moved options inside the function
* removed useless bound
* unused import
* remove useless clone bound.
* make clearer what is secondary node
* same inner manager for main and secondary
* fix example
* enforce minimum and maximum input size for Forkserver's shared memory map and file based testcases
* explicit conversion to OwnedSlice
* clippy
* error if min_input_size > max_input_size when building forkserver
* use Self::State as much as possible
* makeing progress, little by little
* more
* add
* more and more
* more
* more
* mre
* fix
* a
* pp
* fix
* fix
* more
* version
* fix cargo fuzz
* try this thing
* this?
* use new mmap signature in nix.
* fix import
* fix to work with addr2line v0.23.
* remove unused flag
* update strum also for dependency...
* clippy
* clippy
* update lain git hash
* bump many outdated crates
* fix hash
* introduce feedbacks hit tracking for testcases
* make Testcase::hit_feedbacks into Cow<&str> instead of String
rename get_hit_feedbacks to append_hit_feedbacks
update documentation
* simplify ConstFeedback
* rename Feedback::last_result to prev_result
* impl TODO prev_result for NewHashFeedback, ListFeedback, TransferredFeedback, NautilusFeedback
* rename prev_result to last_result
* add docs
* introduce Objectives hit tracking
* update docs
* update Cargo.toml docs
* update docs
* track Feedbacks & Objectives hit in Fuzzer::add_input
* fmt
* clippy
* fix type error in OomFeedback::last_result
* impl last_result for AsanErrorsFeedback
* add track_hit_feedbacks as a feature to libafl_libfuzzer_runtime
* fix clippy
* change return type of Feedback::last_result to a Result
* remove expect in NewHashFeedback::is_interesting
* move Error::premature_last_result to libafl from libafl_bolts
* Make every builder ::builder()
* Fix no_std
* More
* Fix clippy, stuff
* More fun
* Make NopShMem do something
* Alloc
* more fmt
* Remove UB in tinyinst executor builder
* Make builder order not matter for tinyinst
* More better
* fix
* docs
* fmt
* more fmt
* clippy
* fix fixes
* tiny thing
* more betterg
* more more
* more builder
* more builder
* more nyx
* undo breaking clippy
* clip
* Factor out `StageStack` from `StdState`
Refactoring for the sake of abstraction and maintainability.
Previously, the `stage_idx_stack` and `stage_depth` fields of
`StdState` had an implicit relationship to one another. This commit
reifies this relationship into a new `struct`. Its fields are hidden
from everything else in `StdState`. It `impl`s `HasCurrentStage` and
`HasNestedStageStatus`, which `impl`s now no longer have mutable access
to the whole state. The `impl`s for `StdState` just forward to these
ones.
Hopefully, this refactoring would make it easier to `impl State` for
other types in the future.
* Create `StageIdx` newtype
Like `CorpusId`, this is just a thin wrapper around `usize`.
* unused import
* Rename a few functions to be more about indices
* Rename Index to Id, fmt
* Move StageId around
---------
Co-authored-by: Langston Barrett <langston.barrett@gmail.com>
* add peek function to Corpus
* send 0 as next peek in NopCorpus
* rename peek to peek_next_free_id
* fix clippy
* add peek_next_free_id to libafl_libfuzzer
* impl peek_next_free_id for ArtifactCorpus
* Add BytesSubMutator that allows us to mutate sub-parts of a bytes-backed input
* no_std
* fix string mutator
* make build
* Fix clippy on macOS
* Docs
* More docs
* Better docs
* --amend
* Renamed bsi to sub_input. Too much BSI
* More more
* balance backticks
* Make splicing usable with sub_input (not that it makes sense)
* More annotations
* more input annotations?
* Implement HasMutatorBytes for &mut Vec
* clippy
* Use a wrapper type instead
* Add wrapper type for Vec as well
* Remove the duplicate BytesInput... lol
* Hook fix
* Implement deinit to pass tests
* Fix aarch64 errors as well as signficantly speed up startup on macos
* fmt
* Fix cmplog_rt
* Fix windows ci
* ci fix 2
* Fix clippy
* Revert hook implementation to new impl
* Add nolinkage option for hooks and fix some comments
* fmt
* Move hooking mechanism to entirely static mut
* fmt
* Fix clippy
* fix windows
* fmt
* Overall the function hook macro to ensure that the original function is correct and not the same function in a different library. Also change static muts to OnceLock
* Fixed infinite loop on SyncDiskStage
* cargo fmt
* added debug messages
* Add left to sync to metadata. Replace HashSet with Vec
* added check on state metadata to not crash in case it's None (start fuzzing without files to sync)
* fmt & clippy
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* add AFL stage names for calibration, colorization, power and sync stages
* clippy
* add missing name field in sync stage
* use consts instead of hardcoding in functions.
change set_name to with_name for PowerMutationalStage
remove irrelevant fn transforming
* make AFL++ name default for all stages
* WIP: windows frida
* frida-windows: fix hooks not present on windows
* windows: allow building using cargo xwin
* frida-windows: fmrt
* frida-windows: cleanup and allow asan/drcov on windows
* frida-windows: fmt
* frida-windows: fix clippy
* frida-windows: handle unknown exceptions gracefully
* frida-windows: rework shadow mapping algo
* frida-windows: add hook functions
* frida-windows: hook functions; fix stack register
* minibsod: enable for windows
* check_shadow: fix edge casees
* asan_rt: rework and add hooks for windows
* inprocess: add minibsod on windows
* Fix warnings
* minibsod: disable test on windows
* WIP: HookRuntime
* Cleanup after merge
* Bump frida-gum version
* Fix conflict marker; update frida
* Make winsafe windows-specific
* Fmt
* Format
* Better detection of clang++ (using cc)
* Make AsanErrors crate public so we can use it in tests
* Add helper to get immediate of operand
* Use HookRuntime to hook asan functions
Tests now passing
* fmt
* Implement recurisve jmp resolve
* Fix reversed logic
* windows_hooks: Don't die if functions are already replaced
* Allow utils to work on windows
* Enable allocator hooking on windows
* Warnings; add trace to free
* Make ASAN tests run windows (with cargo xwin compilation)
* Fmt
* clang-format
* clang-format
* Add more tests
* Fix partial range access bug in unpoisoning/shadow_check
* Merge main
* Fix check_shadow and implement unit tests
* Fix hooking and PC retrieval
* WIP: Working gdiplus fuzzing with frida-ASAN, no false positives
* LibAFL Frida asan_rt and hook_rt fixes for frida_windows (#2095)
* Introduce aarch64
* MacOS fix - MemoryAreas is broken on MacOS and just loops
* Introduce working aarch64 ASAN check
* Implement large blob
* Fix hook_rt for arm64
* Fix poison/unpoison
* Fix shadow check
* Update x86-64
* Fix aarch64 unused import
* Remove extraneous println statement
* merge main
* Fixes
* alloc: add tests, pass the tests
* HookRuntime before AsanRuntime, and don't Asan if Hooked
* hook_rt: Fixes
* Frida windows check shadow fix (#2159)
* Fix check_shadow and add additional tests
* add some additional documentation
* Revert to Interceptor based hooks
* fixes
* format
* Get rid of hook_rt; fixes
* clang-format
* clang-format
* Fix with_threshold
* fixes
* fix build.rs
* fmt
* Fix offset to RDI on stack
* Fix clippy
* Fix build.rs
* clippy
* hook MapViewOfFile
* fmt
* fix
* clippy
* clippy
* Missing brace
* fix
* Clippy
* fomrrat
* fix i64 cast
* clippy exclude
* too many lines
* Undo merge fails
* fmt
* move debug print
* Fix some frida things
* Remove unused frida_to_cs fn for aarch64
* name
* Don't touch libafl_qemu
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Sharad Khanna <sharad@mineo333.dev>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
* QEMU generic memory iterator + Refactoring
* Generic Memory Iterator (systemmode only for now): It is now possible to iterator over memory ranges, independently of the address kind
* Refactoring or Emulator / Qemu structures: they are now handled separately in different files
* Refactoring of Exit Handlers: Result / Error structs have been clarified
* Simple handler for signals
* add new `check-cfg` calls for libafl qemu
* Allow for more frequent updates of TOML monitor
* Don't skip first client
* Reduce code duplication
* Immediately write first TOML file
* Rust fmt
* Use same client numbering as other monitors
* Fmt
* rands: simpler rand_trait code, remove useless test
* rands: provide and use proper random_seed()
* rands: add missing golden tests
* Don't use current_nanos() for seeding
* rands: remove RandomSeed trait
* Generalize `AsIter` to allow iterating over contents of a `RefCell`
Towards `MapObserver`s in safe Rust.
* Helpers for `RefCellValueObserver`
* MapObserver: Return owned Self::Entry from .get()
`Self::Entry` is `Copy`, so there's not much value in returning a
reference from `get()`. Futhermore, returning a reference limits the
possible implementations of `MapObserver`, because it forces the
borrow/reset to outlive the body of the method.
* MapObserver: Replace `.get_mut()` with `.set(idx, val)`
Like the previous commit, this is intended to expand the possible
implementations of `MapObserver` to types with interior mutability,
which can't necessarily loan out their content.
* Make `RefCellValueObserver` into a safe `MapObserver`
* as iter mut
* as slice (mut): allow for non-& refs
* nostd
* CI round 1
* cleanup + AsSlice defs for RefCellValueObserver
* clippy fixes
* avoid unnecessary imports
* whoops, too aggressive
* use deref instead of as slice
* whoops
* fix as slice conditional importing in stable
---------
Co-authored-by: Addison Crump <addison.crump@cispa.de>
* match by ref fix
* impl Named for Reference
* rename
* magic indexing
* whoops
* docs, clippy
* some additional CI complaints
* other libafl_qemu fixes
* missed an alloc feature
* a smattering of fixes
* use from not direct construction
* tinyinst fix
* horrible double-mutability things
* fixup nyx
* from not new
* forkserver_simple fixes
* dogfood: forkserver
* mmmm yummy dogfood
* round one CI fixes
* clippy appeasement
* deref generic impl to simplify usage
* adaptive serialization (ouch)
* remaining clippy items
* I am tired
* new not with
* fixup: aflpp tracing was not actually constructable
* fix tmin
* reduce complexity of map feedback now that we do not need to constrain
* frida fixes
* fix concolic
* type_ref => reference
* Architecture-specific stuff is now in the `arch` directory
* Helpers are now in the `helpers` directory
* `emu.rs` has been moved as `emu/mod.rs` for consistency with the rest of the repository
* introduce load_initial_inputs_disallow_solution to exit if a solution is found during seed loading
* fmt
* rename CorpusError to Corpus
* add LoadConfig to simplify configuration for loading initial inputs
* Rename Error::Corpus to Error::InvalidCorpus
Add documentation to LoadConfig struct
fix nostd for LoadConfig
---------
Co-authored-by: aarnav <aarnav@srlabs.de>
* rands: use splitmix64 for seeding
Seeding with splitmix64 is a good way to avoid starting with
low-entropy PRNG states, and is explicitly recommended
by the authors of both xoshiro256++ and Romu.
While at it, give the xoshiro256++ PRNG its proper name.
* rands: use fast_bound() to generate number in range
* rands: add top-level choose()
* rands: add Rand::next_float()
* rands: add Rand::coinflip() helper
* libafl: unbreak tests that relied on direct seeding
* rands: add SFC64 PRNG
SFC64 is a well-established and well-understood PRNG designed by
Chris Doty-Humphrey, the author of PractRand. It has been tested
quite a lot over the years, and to date has no known weaknesses.
Compared to xoshiro256++, it is slightly faster and is likely to
be a more future-proof design (xoshiro/xoroshiro family of generators
come with quite long history of [flaws][1] found over the years).
Compared to Romu, it is slightly slower, but guarantees absense
of bias, minimum period of at least 2^64 for any seed, and
non-overlapping streams for different seeds.
[1]: https://tom-kaitchuck.medium.com/designing-a-new-prng-1c4ffd27124d
* run qemu fuzzers (qemu_systemmode only for now) in self-hosted runners
* Remove qemu-related fuzzers to general fuzzers
* fix
* Install dependencies before anything else
* Do not use sudo
* Install sudo
* Revert "Install dependencies before anything else"
This reverts commit 107addad5d9f68dec5a9af50831112cd72c28f4d.
* added qemu specific prerequisites
* add -y flag
* Format with nightly
* Do not use nightly only.
Install fmt and clippy for stable as well.
* Install qemu-img for qemu
* fix qemu-img install
* apt update
* Changed timeout.
* Fix qemu_systemmode test
* fmt
* clippy + decorrelate build and run for qemu_systemmode.
* fix fuzzer
* clippy
* add sqlite3-dev to package prerequisites.
* add arm-none-eabi-gcc
* fix profile dir
* fix condition.
* Run less QEMU stuff faster
---------
Co-authored-by: Toka <tokazerkje@outlook.com>
* speed up cargo check
* fix
* composite
* plural
* wrong name
* go brrrrrrrr
* bash
* aaaaa
* why not even more?
* more composite
* fix
* miri
* test for llvm >= 16
* v2
* v2
* naaaa
* JUST MORE INSTANCES
* aa
* fix
* no bash for windows
* reduce a bit as some guys are delayed
* no deps
* Build cargo-fuzz end to end
* Workaroudn failing CI
* Fix toml
* Build
* Exclude from workspace
* Fix version
* Fix workspace
* Fix workspace
* FIx workspace
* Fix path
* Set path correctly
* Install llvm-tools
* Test with distro llvm installed
* Use build
* complete
* remove from includes
* add back fuzzer tests, format
* add test for whether all fuzzers are actually tested in pipeline
* whoops
* I hate CI I hate CI I hate CI
* revert removal of llvm
* sh compatibility
* sigh
---------
Co-authored-by: Max Ammann <max@maxammann.org>
* Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2…"
This reverts commit c68b3816fb680b635f99c337f78185e699864705.
* all
* more
* add
* add
* Revert "Revert splitting libafl_edges_map_max and libafl_edges_map_in_use (#2…"
This reverts commit c68b3816fb680b635f99c337f78185e699864705.
* all
* more
* MapObserver implements Hash
* Rename the hash utility function (in MapObserver) to hash_easy
* Use hash_slice as a helper function to impl Hash trait
* define_python_map_observer macro implements Hash trait
* Also rename hash_easy to hash_simple
* Rename hash_slice to hash_helper
* hash_helper is used to define the implementation of hash function/trait
* Factor out the Hash trait and function for runtime library structs (#1977)
* Simplify hash_simple (of trait MapObserver) (#1977)
* Use hash_one function to make hash_simple a one-liner
* remove hash_helper
---------
Co-authored-by: Edwin Fernando <ef322@ic.ac.uk>
Co-authored-by: Addison Crump <addison.crump@cispa.de>
* LibAFL QEMU can now be dynamically linked
* LibAFL QEMU reconfiguration happens less frequently (now using a signature check)
* Possibility to have custom rpath in QEMU
* sample implementation of tracking enforcement (incomplete)
* helpful compiler output
* make it look like a real compiler output
* ensure that the macro may be used outside of libafl
* separate index/novelty tracking funcs
* default const generic values so that we don't need to change this everywhere
* fix tests
* rollback unnecessary specification of stdmapobserver
* register metadata in doc tests
* doc fixes
* doc cleanup
* doc cleanup 2
* reduce implementor overhead to zero
* renaming/docs fixes
* asref isn't reflexive??
* generalization stage updates
* add better documentation about require_{indices,novelties}_tracking
* remaining generic updates
* round one CI pass (knowingly introduces breaking changes)
* typo
* round 2 clippy
* rollback: libafl_frida changes
* fmt
* moar porting
* fix remaining fuzzers
* fix windows build, maybe
* fixup libafl_libfuzzer
* fmt nighlty all the things
* attempt to fix some broken additions
* fix fmt
* oops
* fix new invocation
* minimizer scheduler fixes
* fix accounting
* rename
* fix
* Fix build
* Sort generics
* Move more generics into the right place
* Rename A -> C
* Fix test
* Fix test some more
* Fix doc some more
* critical formatting
* More A->C
* CanTrack harder
---------
Co-authored-by: Dominik Maier <dmnk@google.com>
* fix(libafl_qemu_build): assert command success
* fix(libafl_qemu_build): make sure linker_interceptor.py picks up correct compiler
Currently linker_interceptor.py uses 'cc' as the __LIBAFL_QEMU_BUILD_CC environment variable is never set
* remove redudant arg
* Implement user-space QEMU ASAN
* Fix wrong cfgs
* fmt
* merge conflicts in libafl qemu
* A few more fixes to qemu_launcher
* Change commit of qemu-libafl-bridge
* Fix clippy in qemu_launcher
* Fix commit id again
* Empty commit to trigger CI
* Fix path to fuzzer for test in qemu_launcher?
* Revert location of target binary and show the full error log from qemu_launcher test
* Appease the clippy gods
* Empty
* Fix format
---------
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* introduce disabled field to Testcase
* separate executor's processing of execution (adding to corpus/solution/discarding) and execution of input
* introduce add_disabled_input function
* enable splicing mutators to fetch disabled inputs
* reset modified example
* clean up
* update docs
* update docs for count_with_disabled
* fix random_corpus_id for splicing mutator not considering disabled entries
* fmt
* update docs
* clippy
* fix corpus_btreemap not working
* fix clippy warnings
* fix python bindings
* typo in count_with_disabled implementations
* fix certain splicing mutators not considering disabled inputs
* rename count_with_disabled to count_all
* introduce count_disabled function
* update docs for count_all, count_disabled and count
* * introduce get_from_all and nth_from_all for corpus implementations so get() and nth() do not silently fetch disabled entries.
* remove boolean flag from random_corpus_id which allowed inclusion of disabled ids and make it into a new function random_corpus_id_with_disabled
* update docs
* remove boolean is_disabled flag from corpus::insert and make it into a separate function insert_disabled
* rename do_insert to _insert
* make get_from_all inline for cached and inmemory
* add missing functions implementation for PythonCorpus
prevent writing feedback when adding disabled testcases
* fix nth_from_all overfetching enabled corpus entries
* fix clippy & rename execute_with_res to execute_no_process
* refactor _insert for corpus_btreemap
* make LibfuzzerCorpus and ArtifactCorpus to accomodate disabled entries
* fix typo
* fix missing docs for map field
* fix clippy
* test
* (hopefully) fix CachedOnDiskCorpus using incorrect corpus when caching testcase
* fix typo in inmemory_ondisk leading to fetching disabled entry from enabled corpus
---------
Co-authored-by: aarnav <aarnav@srlabs.de>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* CI: Don't run cargo clean in order to preserve build caches
* CI: run yaml formatter >:)
* CI: try sharing build caches between similar jobs
* CI: it looks like the `ucd-generate` tool is not used anymore
* CI: Sneak in merge queue support :)
* CI: split out `cargo fmt --check` job
* libafl_qemu injections: be more resilient about mapping paths
* ci
* ci
* ci
* aa
* aa
* a
* a
* a
* fix
* no fail fast
* system
* what is this lint...
* a
* clp
* disk
* ok no fastfail
* no qemu for now, reenable monday
* Add option to enabled/disable client stats and fix#1771
* more fix
* fix map_density
* even more fix
* remove need for vec in Aggregator::aggregate
* fix json weirdness - remove individual clients (is that all right? )
* Make pretty
* fix non-generic input in libafl_qemu
* Update setup_libxml2.sh of nyx_libxml2_standalone fuzzer (#1990)
On testing this on my machine not all neccessary files for fuzzing with nyx where provided by the setup. Compared to the nyx_libxml2_parallel fuzzer this packer generation line was missing.
After adding this the fuzzer was able to start.
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Tobias Mayer <arbeitskraftmayer@gmail.com>
On testing this on my machine not all neccessary files for fuzzing with nyx where provided by the setup. Compared to the nyx_libxml2_parallel fuzzer this packer generation line was missing.
After adding this the fuzzer was able to start.
* Added backdoors to portable header file.
* fix arm macros.
* copy `libafl_qemu.h` in target directory.
* Merged all qemu_systemmode examples into one, settable with features.
Automatic building using `Makefile.toml`
* fix typo.
* added test of QEMU systemmode fuzzers.
* replace test by build for now.
* Support `MutatedTransform` in `TMinMutationalStage`.
* Run `MutatedTransformPost` for the replaced testcase.
* Add clone trait bound for `MutatedTransformPost`.
* Return an error instead of using unwrap.
* Added paging filtering.
Reworked address range filtering to fit with new generic code.
* Fix: renamed remaining QemuInstrumentationFilter instances.
* Renamed sync breakpoint to sync exit.
* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.
* sync_backdoor.rs removal.
Formatting.
* Updated `bindgen` and `which`.
Adapting code to work with update.
* fix: reconfigure cleanly if prior configure was interrupted abruptly.
* Enable sanitizers in QEMU during debug.
* Added target-usable files.
* Added breakpoint structure.
* Adapted other files to work with ExitHandler.
* Adapted existing fuzzer to work with new exit handler.
* fix: use get to avoid crashes.
* Updated README to indicate cargo-make should be installed.
* Added QEMU internal exit handler.
* Adapted qemu_systemmode example with new exit handler.
* Fixed fuzzers to work with new exit handler.
* Trying to fix CI (#1739)
* test
* dummy
* dummy
* Added new examples.
* Forgot to add build scripts.
* format
* format
* clang-format
* python emulator adaptation.
* fixed python bindings.
* clippy fixes.
* python bindings.
* fix qemu_sugar.
* fix fuzzbench.
* fixed import issues.
* misc fixes.
* renamed crate.
* Updated x86_64 stub bindings.
* Fixed static naming.
* binding fmt
* clippy
* clippy
* Removed useless return statement.
* removed advice to install cargo-make in individual repositories.
* symcc_update (#1749)
* Remove unused create_anymap_for_trait macro (fixes#1719) (#1752)
* Fix `as_object` UB discussed in #1748 (#1751)
* Fix as_object UB discussed in #1748
* More cleanup, more less UB
* Fix fixes
* Added uninit_on_shmem api
* clippy
* fmt
* trying to fix fuzzers, libfuzzer wrapper
* Add OwnedRefMit::owned constructor, libfuzzer fix
* Some more fixes
* Add BacktaceObserver::owned fn
* fmt
* more fmt
* Ignore SigPipe by default (#1741)
* Ignore SigPipe by default
* Fix no_std
* fmt
* Fix incorrect imports (#1758)
* Fix incorrect imports
https://doc.rust-lang.org/core/simd/trait.SimdOrd.html
* Fix
* Try fix ci
* Documentation fixes (#1761)
* Documentation fixes
* Fix InProcessExecutor url
* Update all urls to latest
* Miri ignores for M1 regex (#1762)
* Enabling DrCov on Windows (#1765)
* Enabling DrCov for Windows
* Dedup common code in scheduler (#1702)
* dedup common code in scheduler
* del eco
* fixing
* fix
* replace `Emulator::new_empty` by `Emulator::get` calls outside of `emu.rs` for safety. (#1763)
* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, and more (#1754)
* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, set_error_print_panic_hook
* Trying to fix#1753
* typo
* More fix
* Fix test?
* more testcase fixes
* Fix: renamed remaining QemuInstrumentationFilter instances.
* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.
* format
* format
* format
* Replace sync_exit with sync_backdoor.
* Rework command system.
* fix bad import.
* format.
* cargo fmt
* disable af-xdp as well to avoid linking errors.
* End of merging.
* format.
* Adaptation for usermode.
* format.
* injection support.
* usermode fixes.
format.
* clippy
* clippy + format
* Do not unwrap emu + format.
* fix: entry_point breakpoint
* inital commit.
* clippy
* tests
* clippy
* adapt example
* systemmode.
* renaming
* fmt
* fix lints.
* more lint fix.
* even more lint fixes.
* always more lint fixes.
* lint fix.
* allow unused qualifications for crate when it could be confusing.
* Still lint fixes.
* Lint fixes on generated code.
* Some lint fixes.
* merge continue.
* renamed modules as well.
* fixing merge.
* systemmode compiling.
* fmt
* fix early emulator drop.
* fmt
* fix cast to c_void of the wrong object.
* Added global enum for snapshot managers.
Some renaming.
* move things around.
* WIP: generic inclusion of QEMU Executor in exit handler.
* * Moved extern calls to `libafl_qemu_sys`
* Replaced old `Emulator` by `Qemu` and only kept C functions wrappers
* Now `Emulator` is for higher-level interactions with QEMU. Kept old functions for compatibility calling to `Qemu` functions
* A direct side effect of this slit is the removal of the `IsEmuExitHandler` trait dependency added in many parts of the code.
* Removed old dirty casting for `QemuExecutor` helpers and used the brand-new access to `QemuExecutorState` instead.
* Minor changes to `Qemu` and `Emulator` `get` methods for cleaner getters.
* Add missing `Qemu` function.
* Updated `qemu_systemmode` example.
* Adapted QEMU fuzzers + renaming.
* Fixed python.
* fix libafl_sugar with new implementation.
* fix dangling RefCell.
adapt new examples.
TODO: merge `libafl_systemmode.*` examples.
* clippy.
* fix more fuzzers.
* clippy.
* Implement `HasInstrumentationFilter` generically.
Updated `StdInstrumentationFilter` accordingly.
* Renamed breakpoint functions for QEMU.
`qemu.run()` handling.
* Removed OnceCell / RefCell in signature.
more explicit `MmapPerms` method names.
* minor code refactoring
* Emulator::run_handle refactoring
* deprecated Emulator functions calling directly to QEMU functions.
* IsSnapshotManager -> SnapshotManager
* IsEmuExitHandler -> EmuExitHandler + fmt
* Generic register when it makes sense.
* reverted IsSnapshotManager -> SnapshotManager because of a collision.
* fix syntax + clippy
* fmt
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: lazymio <mio@lazym.io>
Co-authored-by: Bet4 <0xbet4@gmail.com>
Co-authored-by: mkravchik <mkravchik@hotmail.com>
* Make fuzzers/tinyinst_simple support Linux (#1316)
Fix a documentation error for `MmapShMemProvider`.
* Support shmem for `fuzzers/tinyinst_simple` on Linux.
Format code.
* Fix CI to install cxxbridge-cmd.
* Add `CARGO_TARGET_DIR` in makefile to fix CI.
`auto.postcard` file is following `postcard@0.7` wire format, which is
not compatible with the `postcard@1.0` dependency. `auto.json` is out of
date with the latest `Trigger` struct definition. Regenerated both using
the data from `auto.postcard`
* fix(Dockerfile): add build_linux.rs to libafl_qemu and libafl_qemu_sys
* fix(Dockerfile): workaround for llvm.sh failing on bookworm
Revert this once https://github.com/llvm/llvm-project/issues/62475 has been resolved
* fix(libafl_qemu): add unsafe to callbacks
* chore(devcontainer): update devcontainer to work with newer VSCode versions
* revert(libafl_qemu): revert unsafe keyword
* fix(devcontainer): move settings and extension under customizations.vscode
* fix(devcontainer): use postCreateCommand instead of initializeCommand
* Move into_vec to extra trait
* fix no_std
* First step towards stages tuples as vecs
* Allow dyn in StagesTuple, add Current Testcase API, un-traitify Progress, cleanups
* Move from generics to impl keyword, more replacements with better API
* rename fn
* Fix additional stages, more cleanup, rename progress to retries
* Fix more fixes
* Fixes
* Rename ProgressHelper -> RestartHelper
* Fix sugar, python, add perform_restartable
* fixes
* remove prelude bs
* rename to restart_progress_should_run
* more cleanup, remove tests I don't understand (sorry)
* fix docs
* more fix
* fix miri
* unsafe safety annotations
* more comments
* last docs
* Mixed_attributes only allowed for bindgen
Add `--locked` flag when installing `sccache` in the first stages to ensure the image gets built when the base image's Rust version satisfies `sccache`'s MSRV.
Fixes#1923🐛
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* introduce multicore load initial corpus
* update fuzzers/libfuzzer_libpng_norestart to use multicore corpus loading
* run clippy
* use CoreId and Cores in state if std
* misc. typos
* adapt multicore load initial inputs to allow resumable corpus loading
in case of crashes or timeouts during corpus loading.
* add std feature flag to multicore_inputs_processed
* fix doc comment
* run fmt for example fuzzer
---------
Co-authored-by: aarnav <aarnav@srlabs.de>
Co-authored-by: Romain Malmain <romain.malmain@pm.me>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Cargo previously complained when loading libafl as a git dependency:
warning: skipping duplicate package `fuzzbench` found at
`~/.cargo/git/checkouts/libafl-c33dc6f5ec2f7a70/55a300d/fuzzers/fuzzbench`
* fix(Dockerfile): allow running commands with docker run
According to
<https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2109#note_47480476>
adding the "-c" to the ENTRYPOINT is necessary to be able to run
commands like `docker run libafl "cargo build"`
* fix(Dockerfile): restore ability to docker run without command
As `bash -c` always requires a string, the previous commit
removed the ability to just `docker run -it libafl` and
get a shell.
This commit restores that ability
* inital commit.
* clippy
* tests
* clippy
* adapt example
* systemmode.
* renaming
* fmt
* fix lints.
* more lint fix.
* even more lint fixes.
* always more lint fixes.
* lint fix.
* allow unused qualifications for crate when it could be confusing.
* Still lint fixes.
* Lint fixes on generated code.
* Some lint fixes.
* renamed modules as well.
* Fix for #1881, ProbabilitySamplingScheduler seems to be inverted
* Run 'cargo +nightly fmt'
* Added debug_assert as negative and infinite probability values would break the current implementation of next()
---------
Co-authored-by: Dan Blackwell <danblackwell@eduroam-int-dhcp-97-192-218.ucl.ac.uk>
Co-authored-by: Dan Blackwell <danblackwell@ML-W0NYVW4XCK.local>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* fix lints.
* more lint fix.
* even more lint fixes.
* always more lint fixes.
* lint fix.
* allow unused qualifications for crate when it could be confusing.
* Still lint fixes.
* Lint fixes on generated code.
* Some lint fixes.
* Implement MutatorTuple for Vecs to allow Dynamic Mutator Choices
* fix test
* clippy
* Move into_vec to extra trait
* fix no_std
* more nostd
* no_std
* do that again but smarter
* remember to register
* appease the clippy
* cleanup
* autofix clippy
* more clippy fixes
* more clippy...
* small clippy fix
* with_tries => with_retries
* most recent suggestions
* final clippy... hopefully
* libafl_targets: on macOS, do not provide a default implementation for weak functions
* libafl_libfuzzer: update README to talk about macOS specifics
* libafl_targets: allow __sanitizer_cov_pcs_init to be called more than once
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Fixing the mixer according to the xxHash implementation
* Using Rust xxhash-rust instead of handwritten implementation
* Fixing the mixer according to the xxHash implementation - linking directly, not relying on bolts option
* clippy+fmt
* Removing rotation, as it causes issues
* Not using xxhash directly, relying on bolts
* Fmt
* safe linking initial commit
* Fix static declaration.
* Works also for static variables, even for thread-unsafe types.
* Remove warnings.
* fixed visibility for function.
* remove `rustversion` from dependencies.
* move windows, inprocess fork to a different file, try new hook mechanism for the executor
* fix
* even more
* more
* more
* fix
* fix
* macosgit add -ugit add -u
* windows!
* windows!
* aa
* aa
* macos
* std
* wtf unresolved?
* Copy, Clone
* why you just don't have the same API!
* inproc
* next; inprocess
* windows?
* ci
* ci
* ci
* unused
* ci
* unused
* no_std
* windows no std
* fix
* inprocess
* fix
* windows
* fuzzers
* macos , book
* fix
* aa
* allow
* fix
* stop suggesting wrong lint AAAAAAAAAAAAAAAAA!!!
* stop suggesting wrong lint AAAAAAAAAAAAAAAAA!!!
* win
* fix
* wip
* wip2
* windows done?
* remove TimeoutExecutor
* ci
* ci
* miri
* fixfi
* compile on windows
* a
* clp
* no_std stuff
* windows no_std
* mac stuff
* m
* a
* ci
* ci
* deleting timeoutexecutor, gradually
* fucking macos
* ci
* test
* ci
* ci
* batch mode constructor
* fix
* ci
* aa
* miri
* aaa
* tmate again
* fix windows stuff
* final fix
* another win fix
* add
* let's add the new fix later
* more
* fi
* parse
* win clippy
* win no std
* safety
* fix
* DEFAULT
* final fix
* libafl_libfuzzer
* comments
* fix
* fix fuzzres
* fixxxxx
* fixxxxx
* last fix
* change name
* POC attempt to make cmplog work on x64
windows POC seems working
unix POC seems working :)
* no register collisions
* rsp-related ref support
iced optional dep
iced depends on cmplog
warnings
one more warning
comments cleanup
ci unbreak
rebase windows unbreak
rebase unix unbreak
unix only
fmt check
clang formatting
clang formatting again
make clippy happy
formatting
double import
windows unbreak
hashmap is conditional
leftover definition
tutorial related formatter
review fixes
comments
.asm fuzz targets for cmplog on Windows
more tests
rip-relative reference support without index register form
proper ignore rip-related references and ignore 8 bit comparisons
another try_into packing
* harness modification reverted
* dummy commit to restart CI
* review comments
---------
Co-authored-by: sbarsky <sbarsky@denuvo.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* First draft of a Asan tests. As of now, unix-only. This is a WIP, as 1) destroying Gum causes segmentation fault and thus a single test is supported by using a static Gum object. Ideally, this should be fixed and a new Gum instance would be created for each test. 2) 70 identical errors are reported by Asan instead of a a single one. Apart from that, the draft fixes a number of errors found in Asan
* Fmt fixes
* PR comments addressed
* Not crashing upon Asan errors while testing
* More PR comments: removing env_logger, renaming harness to test_harness
* Revert "More PR comments: removing env_logger, renaming harness to test_harness"
This reverts commit 2d3494b3f56e0a5ef23566cb9a884e8c57867b57.
* More PR comments: removing env_logger, renaming harness to test_harness
* Checking for clang presence and failing the test if harness not found
* Fmt
* Running multiple Asan tests
* Cpp Fmt
* clang-format
* More clippy complaints and Apple compilation
* Last clippy complaints (ran scripts/clippy.sh)
* Fixing unused MacOS function
* Fixing unused MacOS imports
* Clang-format latest
* add
* rdm
* CHATGPT
* CHATDUMB
* different action
* no
* fix
* ci
* ci
* ci
* fuck
* dir
* checkout?
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* don't use shit
* ci
* ci
* ci
* FMT
---------
Co-authored-by: toka <tokazerkje@outlook.com>
* add
* rdm
* CHATGPT
* CHATDUMB
* different action
* no
* fix
* ci
* ci
* ci
* fuck
* dir
* checkout?
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* ci
* don't use shit
* ci
* ci
* ci
* Move Clippy to extra CI steps
* clippy?
* Last try with nightly
* no clue why but hey
* cleanup
* more
* revert
* check ubuntu works
* f
* debug
* clippy_ci
* remove tmate
* regex
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* initial commit: multipart
* document + wrap up baby fuzzer
* oops
* core
* add from method, option to iter
* improve example; use minmap; fix initial_mut
* bindings
* clippy, again
* moar clippy
* fmt
* drop rand dep because we don't need it, actually
* docfix
* ok actually fix docs pls
* add
* ci
* wip
* make type name more clear
* last
* add
* more fix
* chg
* fmt
* save changes
* fix_handler
* cfg
* win
* fix
* toml
* f
* more
* fix all the stuff
* fix
* revert fuzzers/fuzzbench to origin/main
* Made changes to libafl_atheris/src/lib.rs to work with Atheris
* Removed unused imports
* Ran on libafl_atheris
* Fixed issue with using extra_counters and issue with using println
---------
Co-authored-by: eceo902 <evan.seeyave@gmail.com>
Co-authored-by: Evan <53020947+eceo902@users.noreply.github.com>
* Partially finish ASAN and CmpLog changes
* Fix handle_trap, report_error, and remove capstone
* Fix a few bugs. Can now detect UAFs properly
* Some small changes
* Make API more consistent with x86
* Fix printing
* Remove unneeded inputs, final changes
* formatting
* Fix x86 build
* Formatting
* Adding support for shutdown upon Ctrl+C on Windows for LLMP
* PR comments and clippy suggestions addressed
* Enable CI for PR branches and manually triggered CI
* Removed an empty line that broke compilation on some platforms
* Trying to fix nostd compilation
* Trying to fix nostd compilation for nightly toolchain
* Removing use that is unused on some platforms
* Trying to fix build on the nightly toolchain
* Trying to fix build on the nightly toolchain, take 2
* Unifying LlmpShutdownSignalHandler
* Fmt fix
* Making the handler pub(crate)
* Nightly toolchain fmt fixes
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Fix: typo in variable name.
* Fix: thread-safe static for emulator initialization.
* Initial support for synchronous exit from QEMU.
* New commands for the sync exit feature.
Supports physical and virtual address requests.
Updated for new SyxSnapshot naming.
* update qemu commit and fix some things
* - Removed lazy_static dependency
- Compiles for usermode
- Format
* Fix warnings
* Fixed sync_exit for missing architectures
---------
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* check
* clippy fmt fixing all the stuff
* restore Cargo.toml
* a
* ci
* ci
* a
* a
* workging?
* work
* ?
* why it worksgit add -u
* ci
* ci
* TMATE
* ci
* ci
* ci
* remove tmate
* less
* fuck; let's try with introspection first
* fucking macro
* another windows shit
* stop it
* i'm harassed by how shit windows is
* fixing
* ci
* ziopera
* fix from main
* ci
* ci
* create the string classification stage
* modify API to pre-group
* preserving mutator
* more meaningful test
* subproperty mutators + some fixes
* document, finalise, integrate with libafl_libfuzzer
* add example, fix for weird range select
* fix for introspection
* fix fuzzer build
* speed optimisation: allow, but do not require, stacking
* property => category
* token replacement
* fixup: rare case where rust does not agree on valid character
* fix CI again
* again again
* take two: dynamic unicode discovery
* oops
* fix: last byte is never selected
* opt: bias to smaller unicode categories
* fix test
* opt: precompute regions and fix tests
* cache and allow stacking
* document and update libafl_libfuzzer
* oops, use reverse
* fix bolts clippy error
* fixup part 2
* clippy
* part 2
* clippy warning allow
* clippy complaint
* use alloc not std
---------
Co-authored-by: toka <tokazerkje@outlook.com>
* refactor: Remove Debug supertraits
Instead of having the Debug trait as supertrait on several traits, the
Debug trait is now required in bounds in specific implementations that
need this specific trait. This keeps the API cleaner, since users now
don't have to propagate the Debug requirement if they don't need to use
the Debug trait.
* refactor: Reformat code
* fix(drcov_rt): coverage files are overwritten if have the same names
Make it unique.
* fix(drcov_rt): use coverage and input as a filename, skip empty covs
* improved libfuzzer corpus
* use .into() for converting ids to usize
* oops
* fix warning about unused arg
* fix some lingering CI errors
* actually save the last lmao
* switch and instruction pass copy paste
* FMT
* add cb
* put things in the functions
* fi
* working
* oops
* options
* no 128bit int on windows
* no 128bit on windows
* oops
* unix only
* windows
* PACKED
* let's stop using it, it's ugly
* fmt
* adjust
---------
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* Call the original QEMU user crash handler in libafl_qemu
* Return if real crash or not
* merge
* Fix singal handlers in libafl and libafl_qemu
* doc and clippy
* clippy
* clippy
* clippy
* slirp
* fix
* fix system
* Revert "Insert into corpus if feedback is_interesting on crash/timeout (#1327)"
This reverts commit 871dfa0a013f31f84e43e125105febca2f137049.
* unused
* Fix issue with libafl_qemu being repeatedly rebuilt
* Changes to make qemu_launcher a production ready fuzzer
* Remove _get prefix
* Don't collect DrCov data during the campaign
* Fix poor performance
* Better validation for core selection
* Changes to print debug when running in verbose mode
* Autofix
* Remove afl++-clang
* Fix build error on 32-bit
* Fix some clippy
* Fix OSX
* Set default version of clang/clang++
* Review changes
* Fix issue with fd sharing between processes
---------
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* endif
* just fix every profile
* WHY YOU DONT JUST USE THE SAME NAME FOR DIR
* fix
* Some fuzzer want release
* WHY I ALWAYS FORGET COMMA
* NO MORE SPACE
* rename
* stb doesn't like debug build
* just use release
* another just use release
* impr(frida): Don't keep FuzzerOptions in Helper
Instead, keep the actual values that are needed. This allows us to make
a builder for FridaInstrumentationBuilder in a subsequent commit.
* refactor(frida): Move workaround to separate method
This is just code movement.
* refactor(frida): move transformer initialization
Mostly code movement here, sets up replacing `new` with a builder. The
one exception is the introduction of a lifetime bound on RT, which needs
to outlive the transformer. This could be generic, but there's probably
no reason to introduce an additional lifetime.
However, because of this lifetime introduction, this is _technically_ a
breaking change.
* impr(frida): Pass module map to runtimes
Instead of passing a slice of modules to instrument, and re-building the
modulemap, pass a Ref-counted module map directly to the initialization.
* feat(frida): Builder for InstrumentationHelper
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* impr(frida/alloc): optional options in allocator
Move all the initialization into Default::default with sensible defaults
and override parameters set from options in new.
* impr(frida): remove options from AsanError
The only option AsanError uses is whether to continue on error. Instead
of keeping a whole clone of the options around, just store that single
boolean value.
* impr(frida/asan): Use less FuzzerOptions
* Implement Default::default to get a good default AsanRuntime
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* update tuneable: consistently access metadata + force 'choice'
* oops, loop in the wrong place
* clarify API some; allow for least of set configuration
* Add an example fuzzer with AFL-Style UI
* fix CI errors
* fix CI and improve the UI
---------
Co-authored-by: toseven <Byone.heng@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* add the metrics(pending,own_finds,imported)
* add the pend_fav metrics
* push
* Add the feature that AFLStats is computed and reported in AFLStatsStage
* fix some cicd errors
* AFLStats migrates to stage/stats.rs
* fix the cicd error
* fix some bugs and resolve the conflicts
* fix some typos
---------
Co-authored-by: toseven <Byone.heng@gmail.com>
Co-authored-by: toka <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* remove unneeded loop in `SpliceMutator::mutate`
previously we searched for the first and the last difference
between exactly the same 2 inputs 3 times in a loop
* remove unused struct fields
* avoid allocating strings for `Transition`s
* avoid allocating `String`s for `Stack`s
* avoid allocating Strings for `Element`s
* apply some clippy lints
* some more clippy lints
* simplify regex
* remove superflous if condition
* remove the Rc<_> in `Element`
* small cleanups and regex fix
* avoid allocating a vector for the culled pda
* bug fix
* bug fix
* reintroduce the Rc, but make it use the *one* alloced VecDeque this time
* slim down dependencies
* use Box<[&str]> for storted state stacks
this saves us a whopping 8 bytes ;), since we don't have to store
the capacity
* revert the changes from 9ffa715c10089f157e4e20563143a2df890c8ffe
fixes a bug
* apply clippy lint
---------
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* implement an AFL-Style TUI
* improve the tui/mod.rs according to the reviews
* fixing fmt manually
---------
Co-authored-by: toseven <Byone.heng@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* Make xxh3 hashing optional (and default to ahash)
* make xxh3 default anyway
* move import
* fix no_alloc
* No ahash without alloc
* fix import
* Keep xxh3 as default for libafl as well
* no randomness for xoshiro
previously we searched for the first and the last difference
between exactly the same 2 inputs 3 times in a loop
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* update documentation of `MinimizerScheduler`
(convert a few references to types into doc links and update the docs of `on_remove`)
* replace manual implementation of `slice::fill`
the in the code comment linked stackoverflow comment https://stackoverflow.com/a/51732799/1345238/
now mentions `slice::fill`
it seems to compile to the same thing as the old version https://rust.godbolt.org/z/98Y4x97vY
* fix docs for the `*InterestingMutator`s and `*ByteAddMutator`s
the macros didn't previously docs gens didn't previously
generate fitting docs
* Make cmp metadata generic, rename ForkserverCmpObservers with more accurate names
* Fix zeroed assignment in cmplogmap
* Dont use prelude in libafl_targets
* Make _mut functions actually return mut references
* Fix fuzzbench forkserver build
* Add type alias for easier construction of the standard cmp observer and add aux data accessors
mostly due std::cell namespace introduction in the 1.70 version.
as rust versions evolve fast enough, it might be easier than
having conditional dependency on he old once_cell crate.
* prep for publishing libafl_libfuzzer
* learn to use linkers
* document-features
* special handling for fuzzbench builds
* Update cmplog.c
* drop dep for llvm-tools; add testcase for memcmp sanity
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* squash libfuzzer edits
* fixup: compat with custom mutators
* use tui flag
* add introspection support
* use libfuzzer dep now that we've merged
* force input loading
* some fixes
* begin docs, impl shrink
* make whole-archive conditional and not default
* make more copies of counters maps
* lol, remember to add the observer
* make size edge map observer an observer
* fixup: make def of run driver conditional
* add sanity checks for insertion
* revert silencing of forks
* add experimental tmin support; add default asan flags
* use default options instead of specifying our own
* implement lockless mode
* fix merge
* fixup lockless corpus
* fixup for generalisation
* remove erroneous drop_in_place
* improve error logging in the case of corpus loading failure
* ok, use lock files 😔
* fix tmin
* implement merge (again); fix rare cases with maps being too small
* implement a scheduler for removing excess
* implement a walking strategy for corpus loading for large corpora
* revert filename parameter; rename and remove duplicates
* various cleanup and clippy satisfaction
* fix no_std tests
* clang-format
* expand and satisfy the clippy gods
* fix sanitizer_ifaces bindgen for no_std
* fix wasm fuzzer
* fixup clippy script
* rename and provide a small amount of explanation for sanitizer_interfaces
* fixup: HasLastReportTime
* fix clippy oddities
* restrict clippy checks to linux-only for libafl_libfuzzer_runtime
* name the mutators
* format
* fix clippy warning
* hope docker is fixed
* fix cmin lint
* clippy pass
* more docs
* more clippy
* fix remaining clippy complaints
* fix import
* miri fixes (no constructors executed)
* exclude libafl_libfuzzer from cargo-hack
* fix clippy check for sanitizer_interfaces
* fmt
* fix CI (?)
* deduplicate sancov 8bit for improved perf on ASAN
* merge 8bit coverage regions + comment out insane deduplication
* no erroring out on free hooks
* fixup for non-forking merge
* skip the corpus dir if we use it
* fixup: recent libafl changes and feature flags
* libafl_libfuzzer: use rust-lld for whole-archive feature
* clarify cause of failure
* mark unsafe
* clippy :cursed_cowboy:
* attempt to fix wasm
* spooky unknowable bug 👻
* more clippy lints
* clippy fix for merge
* use the version pin
* add unsafe to ::register
* Serdeany autoreg fix
* make type assert actionable
* miri fixes
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dominik Maier <dmnk@google.com>
Co-authored-by: Mrmaxmeier <Mrmaxmeier@gmail.com>
* Allow setting the max iterations
* Rename API
* Fix baby_fuzzer_grimoire
* Relax bound
* Also add a new API for transforming
* Revert back grimoire fix
* Revert bound relax
* Make some functions const
* fix isprint
* more const
* move integer_sqrt to bolts, use binary search, use u128 to handle extreme values
* Technically correct
* clippy
* u64 algo
* More test
* cumulative_distribution to in_place
* move calculate_cumulative_distribution_in_place to bolts
* clippy
* Move math stuff to bolts::math
* actually add math
* math?
* For some reason this fixes things, dunno
* fix builds?
* does that help?
* clippy ignores
* more clean clippy
* more cfg_attr
* qemu snapshot little update proposal.
* reeatablishing the TODO since the change does not do it at all
but getting unrelated constant mask unrelated to mappings protection.
---------
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
This commit fixes some issues regarding the `TuneableScheduledMutator`,
which had an edge case for drawing probabilities.
The user is supposed to provide a vector with probabilities that have to
add up to 1.0, but due to floating-point errors, the number can be
sligthly off.
If the sum ends up being slow, there's a chance that we draw a number
that is bigger than it (for example, 1.0), and that would not be present
in the cumulative distribution vectors (either for iterations or
mutations).
The issue is fixed by setting the last value of the cumulative
distribution to 1.0.
This commits adds some validations in the function that calculates the
cumulative distribution function, making sure that the numbers add up to
1, and that they are all between 0 and 1.
The API is now changed so the functions can now return an error instead
of crashing.
The commit also adds some tests, and fixes the `reset()` function.
This commit rewrites the centralized manager to use a secondary broker, as p2p communication is unreliable during frequent restarts. A centralized launcher is introduced too.
* feat: support specify thread id for frida stalker
* fix: thread_id type
* fix: use official repo for frida-gum
* Merged
* Added back missing bolts
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* More less default
* More clippy
* updated rangemap
* Clean up depencdencies
* Undo accidental remove
* Fix
* trying to fix qemu build
* hexagon be gone
despite being present in the headers, getcontext is not implemented in
musl libc, most likely due to the fact it s a deprecated interface.
The only way around is having the third party libucontext apk package
installed.
* Mutators need names (alternative to #1379)
* Signature of MultiMutator shouldn't be the same as the normal mutator
* Named for python, remove mutator for multi_mutator
* fmt
* clippy edition warning
* clippy
* mac_count doc fix, return cleanup
* WIP: fix ub issue in frida fuzzers
* refactor frida helper: remove unused fields
* revert frida-gum bump. Current frida-gum doesn't build on iOS :/
* libafl_frida: silence must_use_candidate lint
this lint is very noisy, and adding #[must_use] to _all_
(even pure )functions seems very excessive to me
* fix clippy
* fix new exciting clippy errors
* fix CASR build errors
* bump casr version
* more clippy whack-a-mole
* allow needless pass by ref mut as it is improperly marked unnecessary
* Add more features to libafl_qemu to remove some of the heavy lifting from the fuzzers
* Refactor qemu_coverage
* Minor tweaks to fix other fuzzers
* Autofix
* Add CallingConvention to write_function_argument
* Replay reverted clippy fixes
---------
Co-authored-by: Your Name <you@example.com>
* Change executor trait to allow \&mut Input
* Add mut inprocess executor
* Add mut inprocess executor
* Format and fix clippy errors
* Fix more clippy errors
* Revert accidental refactoring of InMemoryCorpus
* Add mut versions of all executors that can support it
* Do not persist possible testcase mutation in stages, shadow/differential executors, or corpus minimization
* Fix missing imports
* Fix executor type for missed qemu items
* Add re-exports for mut executors
* Use InProcessForkExecutorMut in QemuForkExecutorMut
* Update BytesInput harnesses to take mutable references
* Update other-input-type-taking harnesses to take mut references
* Clippy fixes
* Feature gate TryFromIntError import
* Fix missed harness input type in baby_fuzzer
* Fix additional clippy issues
* Fix unnecessary hashes on string literal
* Even MORE clippy fixes
* Fix one more clippy issue
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Fixed Forkserver shmem input length, made it configurable
* fix redqueen clippy
* Clippy
* Clippy
* WHY ARE THERE TWO TIMES THE SAME FN
* More clippy
* Change qemu_launcher fuzzer to support multiple architectures and remove qemu_arm_launcher
* Review changes
* Changes to milliseconds
---------
Co-authored-by: Your Name <you@example.com>
* Insert into corpus if feedback is_interesting on crash/timeout
* Use correct import for HasExecutions
* Windows add missing import
* QemuExecutor add HasFeedback
* Windows asan fix
* Add missing call to scheduler.on_add
* Add missing HasExecutions for windows frida
* QemuExecutor missing HasScheduler
* QemuExecutor missing HasCorput
* Add sample fuzzer which collects DrCov coverage for various architectures using QEMU instrumentation
* Fix clippy
* Rename NullCorpus to NopCorpus
* Added support for verbose output
* Attempt to fix clippy again
* Fix remaining defaults to use x86_64 when no arch specified and be more robust handling partial builds
* Make build even more robust against partial re-builds
* Added missing dependencies to workflow, updated README
* Add missing dependencies for i386
* Another dependency
* More dependencies
* Disable tests on OSX
* Add tmate
* Add missing dependencies and symlink header directory
* Tidy up after test so we don't hog all the disk space
---------
Co-authored-by: Your Name <you@example.com>
* ci
* fi
* Revert "fi"
This reverts commit ed298d71057607f019e64d58687273a01d30e260.
* Revert "ci"
This reverts commit 6b65936990143a6069abd56dcbe633ac37be2ede.
* fi
* Clarify setup steps for the baby fuzzer
Specifically:
- Explicitly mention that the dependency path must point to a specific
directory in the cloned repo (and not the root directory)
- Explicitly mention how to manually trigger the panic in the harness
for testing purposes
* Clean up documentation on the baby fuzzer
Since the baby fuzzer chapter of the documentation is done in a
"tutorial", step-by-step fashion, it would be nice to be able to see
where exactly new lines have to be placed in the existing code. To that
end, the code used in the tutorial is moved to snippets (as is done in
the Rust Book), as it allows for much more convenient maintenance of the
snippets, as well as easy hiding of the non-important code on any given
snippet.
Furthermore, a few minor fixes are applied; a typo on a comment and a
missing unsafe block.
* Fix code snippet attributes for baby fuzzer
Specifically:
- Remove unnecessary `compile_fail` attribute
- Add `ignore` attribute to the snippets of the complete baby fuzzer. As
explained in [#1290], it is expected for the baby fuzzer to return a
non-0 exit code, so this should not trigger a failure during `mdbook
test`.
* Fix CLI snippet language
For CLI snippets, the "language" should be set to `console`.
* Remove nested safe block in baby_fuzzer listings
* Add sancov multimap observer to sancov_8bit target
* Undo autofmt of Cargo.toml
* Fix formatting
* Fix import errors under no-default-features, add Safety to counters_maps_observer
* Make observer function no_mangle to allow it to easily be used in a staticlib crate
* Make clippy happy by using export_name instead of no_mangle
* Add observers feature flag and hide counters maps observer behind it
* Fix formatting
* Add CLANG and CLANG_PP env vars
Add CLANG and CLANG_PP env variables for Windows. Resolves issue if clang and llvm-config are not in the same location.
* Just add clang".exe" for windows
User should have llvm-config and clang.exe clang++.exe in the same directory anyways.
* Ran cargo fmt
* qemu: Return errors from Emulator::new instead of asserting
Libraries should not `assert!` except in cases of unrecoverable (library)
programmer error. These errors are all potentially recoverable, and aren't
internal errors in `libafl_qemu` itself.
* Respond to review comments
* Implement CasrAsanBacktraceObserver for dedupe crashes using libCASR and ASAN reports.
* Use casr observer with forkserver executor
* Add casr deduplication for AsanBacktraceObserver
* Add casr deduplication for BacktraceObserver
* Add Stacktrace filtering
* Move init_ignored_frames to constructors
* Add go ignore regexps for BacktraceObservers
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* Fix fuzzers after HasTestcase (#1123)
* Make the trait a trait
* Implement HasTestcase for Corpora
* fix
* fix
* a
* a
* fix
* wasm32
* a
* f
* f
* aa
---------
Co-authored-by: tokatoka <tokazerkje@outlook.com>
* Created macro to get the metadata form State and Testcase
* Expanded the macros for mutable, or not, State and Testcase metadata
* Created functions on traits HasMetadata and HasNamedMetadatato get, mutable or not, metadata
* Created the functions to get metadata
* Added #[inline] attribute and renamed the functions
* Renamed the functions and added #[inline] attribute
* Temporarily added testcase() function
* Added testcase() function
* Changed Ref import to core::cell:Ref
* Added testcase_mut() and renamed occurences of metadata() and metadata_mut()
* Renamed more occurences
* Renamed the metadata() on impl HasMetadata for NopState
---------
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* Allow take the ownership of the BytesInput
* Add must_use as told
* Implement From&Into for better interoperability
* Format code
* Remove into_bytes
* Remove From<&BytesInput> to avoid misuse
* fix libfuzzer_libpng_cmin
* fix libfuzzer_libpng_ctx
* revert libfuzzer_libpng_cmin and check ci by grepping broker stdout result instead
* revert libfuzzer_libpng_ctx and check ci by grepping broker stdout result instead
* add check ci for fuzzers
* add check ci for fuzzers
* add check ci
* add fuzzbench test
* add validation for qemu fuzzer
---------
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
regex is a large crate, and is only used in a few specific spots. Users should
have the ability to avoid this transitive dependency if not using the features
in question.
* Don't use magic string but string constant
* Don't allow to specify multiple input files with different names
* Ensure that the file name for the current test case is unique for every fuzzer currently running
* Add note advising users to choose distinct names for the input file
* Move builder functions to more generic implementation to allow parse_afl_cmdline rewrite
* Rewrite parse_afl_cmdline to reduce code duplication
* Add remark to documentation regarding the program path
* Change behavior to allow the usage of actual AFL command lines, hopefully without breaking existing code
* Rustfmt
* Move generation of unique filename to fs
* Ensure default input filename for command executor is unique per fuzzing process
* Pass the input to the target via stdin, when no input file is specified
Previous solution of passing it via a standard file is useless, as the target does not know to read said file
* Rustfmt
---------
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
* scheduler replace fixes
* oops, no-std
* add
* changes on the fuzzers
* move map feedback history updates to append_metadata
* fixes for python bindings
* learn to clippy
* fix for fuzzer add_input
* clippy fixes for frida
* additional powersched differences
* corrections for bitmap_size
* off-by-one
* I live in a prison of my own creation and clippy is the warden
* clear the novelties map for the situation where is_interesting is invoked, but not append_metadata
---------
Co-authored-by: tokatoka <tokazerkje@outlook.com>
* Use Instant::now instead of duration
* Use Some
* add custom serde for Instant
* fix linter
* only enable TimeFeedback when std flag is enabled
* fix typo
* fix linter std
* cargo fmt
* allow clippy::trivially_copy_pass_by_ref on custom serde serialize function
* allow TimeObserver and Timefeedback for no_std
* cargo fmt
---------
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Reduces total number of packages from 577 to 571 on building with:
`cargo +nightly build --workspace --all-features`
* ahash 0.7 -> 0.8
* Move `AHasher::new_with_keys` to `RandomState::with_seeds` given the
recommendation from: aHash maintainer:
https://github.com/tkaitchuck/aHash/issues/132#issuecomment-1288207069
* bindgen: 0.61 -> 0.63
* c2rust-bitfields: 0.3 -> 0.17
* criterion: 0.3 -> 0.4
* crossterm: 0.25 -> 0.26
* dynasmrt: 1.2 -> 2
* goblin: 0.5.3 -> 0.6
* hashbrown: 0.12 -> 0.13
* nix: 0.25 -> 0.26
* The `addr` arg of `mmap` is now of type `Option<NonZeroUsize>`
* The `length` arg of `mmap` is now of type `NonZeroUsize`
* Requires updating implementers to update `nix` as well
* prometheus-client: 0.18.0 -> 0.19
* Do not box metrics
* Gauges (a majority of the LibAFL metrics) are now i64 types so there
is a small chance of overflow, with the u64 values that LibAFL
tracks, but unlikely to be problematic.
* Keep `exec_rate` as a floating point value
* serial_test: 0.8 -> 1
* typed-builder: 0.10.0 -> 0.12
* windows: 0.42.0 -> 0.44
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Most of the time, fuzzing campaigns are reasonably long. Accordingly, when
using `NewHashFeedback`, you might have good reason to believe that you'll
find thousands (or more) different observations with different hashes. When
the `HashSet` outgrows its capacity, it can cause reallocation, which is slow.
See the following link for more details:
https://doc.rust-lang.org/std/vec/struct.Vec.html#capacity-and-reallocation
* libafl: Remove `{update,clear}_hash` from `ObserverWithHashField`
These methods aren't used by `NewHashFeedback`, so there's no compelling reason
to keep them in the interface. They preclude implementations of
`ObserverWithHashField` that calculcate a hash on-the-fly from a value. For
example, my use-case is to store the stdout of a process, and use
`NewHashFeedback` to only collect inputs that result in new messages on stdout.
Both of these methods are pretty suspicious to begin with - why should other
code be able to update the internal state of the observer? What are the
semantics of `update_hash`? If there are compelling reasons to keep these
methods, let's clarify their intent in the documentation.
* libafl: Return hash by value from `ObserverWithHashField`
This allows implementors of this trait to not store the hash, but rather to
compute it on-the-fly. Since `Option<u64>` is `Copy` (and quite small), and
this method is called once per execution of the target program, this is likely
to have negligible performance impact.
* libafl: Implement `ObserverWithHashField` for `ValueObserver`
This demonstrates the utility of the previous two commits. Now, `ValueObserver`
can be used with `NewHashFeedback`.
* Clippy, move to ahasher
* Oops :)
---------
Co-authored-by: Langston Barrett <langston.barrett@gmail.com>
* tmp
* more
* save
* TODO
* fix
* update to tinyinst on crates
* dep
* fmt
* shmem done
* cpp fmt
* clp
* fmt
* why??
* ver
* more makefile.toml
* windows test
* Update build_and_test.yml
* fix
* a
* install
* fmt
* fix
* only macos and win
* more
* The order matters
* remove
* fmt
* chg
* typo
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* fix multiple subtle bugs with grimoire, mutators, and state
* obey the clippy overlord
* grimoire: skip over token after splice
* remove extraneous length check
* fixup grimoire/generalisation, remove GeneralizedInput in favour of metadata
* additional cleanup
* transformable inputs to solve the grimoire problem
* explicit use of 'transforming' to keep typing compatible with normal usage
* clippy fix
* fixes for nautilus, python
* explicit inlining for reflexive impl
* fix for tutorial
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* fixing linking issue
* insteading of linking lib we dont use. just disable them
* fixing the keyutils problem
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
* Make the kind of solving more clear (follow up on #980)
* Update docs/src/advanced_features/concolic.md
Co-authored-by: julihoh <julihoh@users.noreply.github.com>
Co-authored-by: julihoh <julihoh@users.noreply.github.com>
* forkserver: Add an API to setup the shared memory region for edge coverage
This is inspired from and meant to be similar to afl-cc's instrumentation.
Remove ! return type from __afl_start_forkserver as it returns in several cases.
* Add example fuzzer using LibAFL's forkserver
The fuzzer is instrumented with libafl_cc as well.
Co-authored-by: ergrelet <ergrelet@users.noreply.github.com>
* a few stylistic/grammar changes
* expression
* some wording and a different git clone command
The original `git clone` command did not work for me (permission denied).
* small wording changes
* review
* typo
* neutral
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
due to the call to pushd on line 23 not being directed at packer, the wrong commit was being used and so that resulted in the wrong init.cpio.gz being generated which hangs when trying to run the libxml2 examples however using the right commit (86b159bafc0b2ba8feeaa8761a45b6201d34084f) fixes this problem.
* Add libfuzzer example for window with ASAN
* Fix formatting
* Add link
* Fix cpp format
* Skip windows fuzzers
* Fix format
* Fix testing fuzzer
* Fix taks name
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
* disable libafl's default features in libafl-frida
to allow users to disable default features while using stuff from libafl-frida
* Update Cargo.toml
Co-authored-by: Dominik Maier <domenukk@gmail.com>
For bugs, feel free to open issues or contact us directly. Thank you for your support. <3
## Pull Request Guideline
Even though we will gladly assist you in finishing up your PR, try to:
- keep all the crates compiling with *stable* rust (hide the eventual non-stable code under [`cfg`s](https://github.com/AFLplusplus/LibAFL/blob/main/libafl/build.rs#L26))
- run `cargo +nightly fmt` on your code before pushing
- check the output of `cargo clippy --all` or `./scripts/clippy.sh` (On windows use `.\scripts\clippy.ps1`)
- run `cargo build --no-default-features` to check for `no_std` compatibility (and possibly add `#[cfg(feature = "std")]`) to hide parts of your code.
- Please add and describe your changes to MIGRATION.md if you change the APIs.
You can also run ./scripts/precommit.sh to execute checks that will be performed on a PR.
Some of the parts in this list may be hard, don't be afraid to open a PR if you cannot fix them by yourself, so we can help.
### Pre-commit Hooks
Some of these checks can be performed automatically during commit using [pre-commit](https://pre-commit.com/).
Once the package is installed, simply run `pre-commit install` to enable the hooks, the checks will run automatically before the commit becomes effective.
## LibAFL Code Rules
Before making your pull requests, try to see if your code follows these rules.
- Wherever possible, use `Cow<'static, str>` instead of String.
- `PhantomData` should have the smallest set of types needed. Try not adding `PhantomData` to your struct unless it is really necessary. Also even when you really need `PhantomData`, try to keep the types `T` used in `PhantomData` as smallest as possible
- Wherever possible, trait implementations with lifetime specifiers should use '_ lifetime elision.
- Complex constructors should be replaced with `typed_builder`, or write code in the builder pattern for yourself.
## Rules for Generics and Associated Types
1. Remove generic restrictions at the definitions (e.g., we do not need to specify that types impl `Serialize`, `Deserialize`, or `Debug` anymore at the struct definitions). Therefore, try avoiding code like this unless the constraint is really necessary.
```rust
pub struct X<A>
where
A: P // <-Donotaddcontraintshere
{
fn ...
}
```
2. Reduce generics to the least restrictive necessary. __Never overspecify the constraints__. There's no automated tool to check the useless constraints, so you have to verify this manually.
```rust
pub struct X<A>
where
A: P + Q // <-Trytousetheassmallestsetofconstraintsaspossible.IfthecodestillcompilesafterdeletingQ,thenremoveit.
{
fn ...
}
```
3. Prefer generic to associated types in traits definition as much as possible. They are much easier to use around, and avoid tricky caveats / type repetition in the code. It is also much easier to have unconstrained struct definitions.
Try not to write this:
```rust
pub trait X
{
type A;
fn a(&self) -> Self::A;
}
```
Try to write this instead:
```rust
pub trait X<A>
{
fn a(&self) -> A;
}
```
4. Traits which have an associated type (if you have made sure you cannot use a generic instead) should refer to the associated type, not the concrete/generic. In other words, you should only have the associated type when you can define a getter to it. For example, in the following code, you can define a associate type.
```rust
pub trait X
{
type A; // <-Youshould(can)defineitaslongasyouhaveagettertoit.
fn a(&self) -> Self::A;
}
```
5. Generic naming should be consistent. Do NOT use multiple name for the same generic, it just makes things more confusing. Do:
```rust
pub struct X<A> {
phantom: PhanomData<A>,
}
impl<A> X<A> {}
```
But not:
```rust
pub struct X<A> {
phantom: PhanomData<A>,
}
impl<B> X<B> {} // <-DoNOTdothat,useAinsteadofB
```
6. __Ideally__ the types used in the arguments of methods in traits should have the same as the types defined on the traits.
- `EventManager` is refactored to avoid calling function from `Fuzzer`, thus we do not evaluate testcases in `EventManager` anymore.
- Now we have `EventReceiver` in `events` module, and `EventProcessor` in `fuzzer` module.
- `EventReceiver` is responsible for receiving testcases and delegates its evaluation to `EventProcessor`.
- `EventProcessor` is responsible for evaluating the testcases passed by the `EventReceiver`.
- Since we don't evaluate testcases in the `EventManager` anymore. `on_fire` and `post_exec` have been deleted from `EventManagerHook`.
- Similarly `pre_exec` has been renamed to `pre_receive`.
- `AsanModule` now uses a `builder()` method for constructing its instances.
- `Monitor` is refactored. Most statistics have been extracted into an individual `stats` module under `monitors`.
- There is a `ClientStatsManager` to manage client statistics, and is owned by `EventManager`. Most of previous `Monitor`'s trait methods have been moved to the `ClientStatsManager`.
- `user_monitor` has been renamed to `user_stats`, `introspection_monitor` has been renamed to `introspection_stats`, perf-related structure definitions have been renamed, and all were moved to the `stats` module.
- `OnDiskTomlMonitor`, `OnDiskJsonMonitor`, `OnDiskJsonAggregateMonitor` are now no longer takes a base monitor to wrap. If you want to use multiple monitors together, simply use a `tuple_list`.
- `MultipartInput` is now implemented as key-value tuples in a `ListInput`. The interface slightly changed, all functionality is maintained.
- Instead of names, `MultipartInput` uses generic `key`s (function names were changed accordingly).
- If you don't need the keys to identify individual parts, consider using `ListInput` directly.
- `StdScheduledMutator` has been renamed to `HavocScheduledMutator`.
## 0.14.1 -> 0.15.0
- `MmapShMem::new` and `MmapShMemProvider::new_shmem_with_id` now take `AsRef<Path>` instead of a byte array for the filename/id.
- The closure passed to a `DumpToDiskStage` now provides the `Testcase` instead of just the `Input`.
- `StatsStage` is deleted, and it is superceded by `AflStatsStage`
- Renamed and changed mapping mutators to take borrows directly instead of `MappedInput`s. See `baby_fuzzer_custom_input` for example usage
- Related: `MutVecInput` is deprecated in favor of directly using `&mut Vec<u8>`
- Related: `MappedInputFunctionMappingMutator` and `ToMappedInputFunctionMappingMutatorMapper` have been removed as now duplicates of `MappingMutator` (previously `FunctionMappingMutator`) and `ToMappingMutator` (previously `ToFunctionMappingMutatorMapper`)
- Related: `ToOptionMappingMutatorMapper` and `ToFunctionMappingMutatorMapper` have been renamed to `ToOptionalMutator` and `ToMappingMutator` respectively
- `Qemu` cannot be used to initialize `Emulator` directly anymore. Instead, `Qemu` should be initialized through `Emulator` systematically if `Emulator` should be used.
- Related: `EmulatorBuilder` uses a single function to provide a `Qemu` initializer: `EmulatorBuilder::qemu_parameters`. For now, it can be either a `Vec<String>` or a `QemuConfig` instance.
- Related: Qemu's `AsanModule` does not need any special call to `Qemu` init methods anymore. It is now possible to simply initialize `AsanModule` (or `AsanGuestModule`) with a reference to the environment as parameter.
- `CustomBufHandlers` has been deleted. Please use `EventManagerHooksTuple` from now on.
- Trait restrictions have been simplified
- The `UsesState` and `UsesInput` traits have been removed in favor of regular Generics.
- For the structs/traits that used to use `UsesState`, we bring back the generic for the state.
- `Input` is now only accessible through generic. `Input` associated types have been definitely removed.
- `HasCorpus` bound has been removed in many places it was unused before.
- `StdMutationalStage::transforming` must now explicitly state the Inputs types. As a result, `StdMutationalStage::transforming` must be written `StdMutationalStage::<_, _, FirstInputType, SecondInputType, _, _, _>::transforming`.
- The `State` trait is now private in favour of individual and more specific traits
- Restrictions from certain schedulers and stages that required their inner observer to implement `MapObserver` have been lifted in favor of requiring `Hash`
- Related: removed `hash_simple` from `MapObserver`
## 0.14.0 -> 0.15.0
- Removed `with_observers` from `Executor` trait.
- `MmapShMemProvider::new_shmem_persistent` has been removed in favour of `MmapShMem::persist`. You probably want to do something like this: `let shmem = MmapShMemProvider::new()?.new_shmem(size)?.persist()?;`
## Pre 0.9 -> 0.9
- [Migrating from LibAFL <0.9 to 0.9](https://aflplus.plus/libafl-book/design/migration-0.9.html)
LibAFL gives you many of the benefits of an off-the-shelf fuzzer, while being completely customizable.
LibAFL is a collection of reusable pieces of fuzzers, written in Rust, it gives you many of the benefits of an off-the-shelf fuzzer, while being completely customizable.
Some highlight features currently include:
- `fast`: We do everything we can at compile time, keeping runtime overhead minimal. Users reach 120k execs/sec in frida-mode on a phone (using all cores).
- `scalable`: `Low Level Message Passing`, `LLMP` for short, allows LibAFL to scale almost linearly over cores, and via TCP to multiple machines.
@ -22,104 +13,80 @@ feel free to add an AST-based input for structured fuzzing, and more.
- `multi platform`: LibAFL was confirmed to work on *Windows*, *MacOS*, *Linux*, and *Android* on *x86_64* and *aarch64*. `LibAFL` can be built in `no_std` mode to inject LibAFL into obscure targets like embedded devices and hypervisors.
- `bring your own target`: We support binary-only modes, like Frida-Mode, as well as multiple compilation passes for sourced-based instrumentation. Of course it's easy to add custom instrumentation backends.
## Overview
## Core concepts
LibAFL is a collection of reusable pieces of fuzzers, written in Rust.
It is fast, multi-platform, no_std compatible, and scales over cores and machines.
LibAFL is fast, multi-platform, no_std compatible, and scales over cores and machines. It offers a main crate that provide building blocks for custom fuzzers, [libafl](./libafl), a library containing common code that can be used for targets instrumentation, [libafl_targets](./libafl_targets), and a library providing facilities to wrap compilers, [libafl_cc](./libafl_cc). It offers integrations with popular instrumentation frameworks. At the moment, the supported backends are:
+ `SanitizerCoverage`, in [libafl_targets](./libafl_targets)
+ `Frida`, in [libafl_frida](./libafl_frida)
+ `QEMU` user-mode and system mode, including hooks for emulation, in [libafl_qemu](./libafl_qemu)
+ `TinyInst`, in [libafl_tinyinst](./libafl_tinyinst) by [elbiazo](https://github.com/elbiazo)
It offers a main crate that provide building blocks for custom fuzzers, [libafl](./libafl), a library containing common code that can be used for targets instrumentation, [libafl_targets](./libafl_targets), and a library providing facilities to wrap compilers, [libafl_cc](./libafl_cc).
## Building and installing
LibAFL offers integrations with popular instrumentation frameworks. At the moment, the supported backends are:
+ SanitizerCoverage, in [libafl_targets](./libafl_targets)
+ Frida, in [libafl_frida](./libafl_frida)
+ QEMU user-mode, in [libafl_qemu](./libafl_qemu)
+ TinyInst, in [libafl_tinyinst](./libafl_tinyinst) by [elbiazo](https://github.com/elbiazo)
## Getting started
1. Install the Dependecies
- The Rust development language.
We highly recommend *not* to use e.g. your Linux distribition package as this is likely outdated. So rather install
Rust directly, instructions can be found [here](https://www.rust-lang.org/tools/install).
- LLVM tools
The LLVM tools are needed (newer than LLVM 11.0.0 but older than LLVM 15.0.0)
- Cargo-make
We use cargo-make to build the fuzzers in `fuzzers/` directory. You can install it with
```
cargo install cargo-make
```
2. Clone the LibAFL repository with
```
#### Install the Dependencies
- **The Rust development language**
- We highly recommend *not* to use e.g. your Linux distribution package as this is likely outdated. So rather install Rust directly, instructions can be found [here](https://www.rust-lang.org/tools/install).
- **LLVM tools**
- The LLVM tools (including clang, clang++) are needed (newer than LLVM 15.0.0 up to LLVM 18.1.3) If you are using Debian/Ubuntu, again, we highly recommmend that you install the package from [here](https://apt.llvm.org/)
- (In `libafl_concolic`, we only support LLVM version newer than 18)
- Just:
- We use [just](https://github.com/casey/just) to build the fuzzers in `fuzzers/` directory. You can find instructions to install it in your environment [in the Just Programmer's Manual](https://just.systems/man/en/packages.html).
#### Clone the LibAFL repository with
```sh
git clone https://github.com/AFLplusplus/LibAFL
```
3. Build the library using
```
#### Build the library using
```sh
cargo build --release
```
4. Build the API documentation with
```
#### Build the API documentation with
```sh
cargo doc
```
5. Browse the LibAFL book (WIP!) with (requires [mdbook](https://github.com/rust-lang/mdBook))
```
#### Browse the LibAFL book (WIP!) with (requires [mdbook](https://rust-lang.github.io/mdBook/index.html))
```sh
cd docs && mdbook serve
```
## Getting started
We collect all example fuzzers in [`./fuzzers`](./fuzzers/).
Be sure to read their documentation (and source), this is *the natural way to get started!*
You can run each example fuzzer with
```sh
just run
```
cargo make run
```
as long as the fuzzer directory has `Makefile.toml` file.
You can run each example fuzzer with this following command, as long as the fuzzer directory has a `Justfile` file. The best-tested fuzzer is [`./fuzzers/inprocess/libfuzzer_libpng`](./fuzzers/inprocess/libfuzzer_libpng), a multicore libfuzzer-like fuzzer using LibAFL for a libpng harness.
The best-tested fuzzer is [`./fuzzers/libfuzzer_libpng`](./fuzzers/libfuzzer_libpng), a multicore libfuzzer-like fuzzer using LibAFL for a libpng harness.
- [Online API documentation](https://docs.rs/libafl/)
- The LibAFL book (WIP) [online](https://aflplus.plus/libafl-book) or in the [repo](./docs/src/)
- Our research [paper](https://www.s3.eurecom.fr/docs/ccs22_fioraldi.pdf)
- Our RC3 [talk](http://www.youtube.com/watch?v=3RWkT1Q5IV0 "Fuzzers Like LEGO") explaining the core concepts
- Our Fuzzcon Europe [talk](https://www.youtube.com/watch?v=PWB8GIhFAaI "LibAFL: The Advanced Fuzzing Library") with a (a bit but not so much outdated) step-by-step discussion on how to build some example fuzzers
- The Fuzzing101 [solutions](https://github.com/epi052/fuzzing-101-solutions) & series of [blog posts](https://epi052.gitlab.io/notes-to-self/blog/2021-11-01-fuzzing-101-with-libafl/) by [epi](https://github.com/epi052)
- Blogpost on binary-only fuzzing lib libaf_qemu, [Hacking TMNF - Fuzzing the game server](https://blog.bricked.tech/posts/tmnf/part1/), by [RickdeJager](https://github.com/RickdeJager).
- [A LibAFL Introductory Workshop](https://www.atredis.com/blog/2023/12/4/a-libafl-introductory-workshop), by [Jordan Whitehead](https://github.com/jordan9001)
+ [Online API documentation](https://docs.rs/libafl/)
+ The LibAFL book (WIP) [online](https://aflplus.plus/libafl-book) or in the [repo](./docs/src/)
+ Our research [paper](https://www.s3.eurecom.fr/docs/ccs22_fioraldi.pdf)
+ Our RC3 [talk](http://www.youtube.com/watch?v=3RWkT1Q5IV0 "Fuzzers Like LEGO") explaining the core concepts
+ Our Fuzzcon Europe [talk](https://www.youtube.com/watch?v=PWB8GIhFAaI "LibAFL: The Advanced Fuzzing Library") with a (a bit but not so much outdated) step-by-step discussion on how to build some example fuzzers
+ The Fuzzing101 [solutions](https://github.com/epi052/fuzzing-101-solutions) & series of [blog posts](https://epi052.gitlab.io/notes-to-self/blog/2021-11-01-fuzzing-101-with-libafl/) by [epi](https://github.com/epi052)
+ Blogpost on binary-only fuzzing lib libaf_qemu, [Hacking TMNF - Fuzzing the game server](https://blog.bricked.tech/posts/tmnf/part1/), by [RickdeJager](https://github.com/RickdeJager).
For bugs, feel free to open issues or contact us directly. Thank you for your support. <3
Please check out **[CONTRIBUTING.md](CONTRIBUTING.md)** for the contributing guideline.
Even though we will gladly assist you in finishing up your PR, try to
- keep all the crates compiling with *stable* rust (hide the eventual non-stable code under [`cfg`s](https://github.com/AFLplusplus/LibAFL/blob/main/libafl/build.rs#L26))
- run `cargo fmt` on your code before pushing
- check the output of `cargo clippy --all` or `./clippy.sh`
- run `cargo build --no-default-features` to check for `no_std` compatibility (and possibly add `#[cfg(feature = "std")]`) to hide parts of your code.
## Debugging
Some of the parts in this list may be hard, don't be afraid to open a PR if you cannot fix them by yourself, so we can help.
Your fuzzer doesn't work as expected? Try reading [DEBUGGING.md](./docs/src/DEBUGGING.md) to understand how to debug your problems.
## Cite
If you use LibAFL for your academic work, please cite the following paper:
```bibtex
@ -149,11 +116,3 @@ Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in this crate by you, as defined in the Apache-2.0 license, shall
be dual licensed as above, without any additional terms or conditions.
</sub>
<br>
<sub>
Dependencies under more restrictive licenses, such as GPL or AGPL, can be enabled
using the respective feature in each crate when it is present, such as the
This file answers some common questions that arise when you are writing a fuzzer using LibAFL.
## Q. My fuzzer crashed but the stack trace is useless
You can enable the `errors_backtrace` feature of the `libafl` crate. With this the stacktrace is meaningful.
## Q. I started the fuzzer but the corpus count is 0
Unless the initial corpus is loaded with the "load_initial_inputs_forced" function, we only store the interesting inputs, which is the inputs that triggered the feedback. So this usually means that your input was not interesting, or your target was simply not properly implemented.
Either way, what you can do is attach to the executable with gdb and set a breakpoint at where the new edges should be reported. If no instrumentation code is executed, then the problem is in the instrumentation. If the instrumentation code is hit, but still your input is not deemed interesting/stored, then the problem could be that you are not passing the observer/feedback correctly to the fuzzer.
## Q. I started the fuzzer but the coverage is 0
This could mean two things. Perhaps your target was not properly instrumented, or you are not using the correct observer, feedback feature.
In this case, again, what usually should do is to run the fuzzer with gdb and set a breakpoint at where the coverage is recorded (e.g. __sanitizer_coverage_trace_pcguard), and validate that the target is giving the feedback to the fuzzer.
## Q. I started the fuzzer but there's no output
First, verify that your stdout and stderr are not redirected to `/dev/null`. If you get the log, then it should either fall into the previous 2 cases. Either the fuzzer crashed because you didn't have the initial seeds, or the coverage feedback is not working.
## Q. I don't see any output from my fuzzer (println!() or logging)
First, check that you are not redirecting things to `/dev/null` else you will see nothing.
To see the log that you added with `log::trace!();`, you need to initialize the logger (any logger, `env_logger` or `SimpleStdoutLogger` from `libafl_bolts`) before the fuzzer starts.
Also you have to make sure that you are runing with `RUST_LOG=<log_level>` and you are *NOT* using `release_max_level_info` feature of `log` crate in your `Cargo.toml` of your fuzzer
## Q. My fuzzer is slow
Try running the fuzzer with the `introspection` feature of the `libafl`. This will show how much time is spent on each module of your fuzzer. Also you might be using a wrong size of the coverage map. If you see `2621440` for the size of the coverage map, you are doing it wrong. One possible mistake is the misuse of `libafl_targets::coverage::EDGES_MAP`
```rust
let map = StdMapObserver::from_mut_ptr("edges", EDGES_MAP.as_mut_ptr(), EDGES_MAP.len());
```
You should *never* use the `EDGES_MAP`'s size as this is just the size of the allocated size of the coverage map. Consider using something smaller or our default value `libafl_targets::LIBAFL_EDGES_MAP_DEFAULT_SIZE`.
## Q. I still have problems with my fuzzer
Finally, if you really have no idea what is going on, run your fuzzer with logging enabled. (You can use `env_logger`, `SimpleStdoutLogger`, `SimpleStderrLogger` from `libafl_bolts`. `fuzzbench_text` has an example to show how to use it.) (Don't forget to enable stdout and stderr), and you can open an issue or ask us in Discord.
## Q. My fuzzer died of `Storing state in crashed fuzzer instance did not work`
If the exit code is zero, then this is because either your harness exited or you are using fuzzer_loop_for and forgot to add `mgr.on_restart` at the end of the fuzzer. In the first case, you should patch your harness not to exit. (or use `utils/deexit`).
## Q. I can't leave the TUI screen
Type `q` to leave TUI.
## Q. I see `QEMU internal SIGSEGV {code=MAPERR, addr=0x48}` and my QEMU fuzzer doesn't run
Are you running QEMU fuzzer on WSL? You have to enable vsyscall <https://github.com/microsoft/WSL/issues/4694#issuecomment-556095344>.
LibAFL has support for concolic tracing based on the [SymCC](https://github.com/eurecom-s3/symcc) instrumenting compiler.
For those uninitiated, the following attempts to describe concolic tracing from the ground up using an example.
For those uninitiated, the following text attempts to describe concolic tracing from the ground up using an example.
Then, we'll go through the relationship of SymCC and LibAFL concolic tracing.
Finally, we'll walk through building a basic hybrid fuzzer using LibAFL.
@ -92,18 +93,18 @@ In hybrid fuzzing, we combine this tracing + solving approach with more traditio
The concolic tracing support in LibAFL is implemented using SymCC.
SymCC is a compiler plugin for clang that can be used as a drop-in replacement for a normal C or C++ compiler.
SymCC will instrument the compiled code with callbacks into a runtime that can be supplied by the user.
These callbacks allow the runtime to construct a trace that similar to the previous example.
These callbacks allow the runtime to construct a trace that is similar to the previous example.
### SymCC and its Runtimes
SymCC ships with 2 runtimes:
SymCC ships with 2 runtimes:
* a 'simple' runtime that attempts to solve any branches it comes across using [Z3](https://github.com/Z3Prover/z3/wiki) and
* a [QSym](https://github.com/sslab-gatech/qsym)-based runtime, which does a bit more filtering on the expressions and also solves using Z3.
* A 'simple' runtime that attempts to negate and analytically solve any branch conditions it comes across using [Z3](https://github.com/Z3Prover/z3/wiki) and
* A [QSym](https://github.com/sslab-gatech/qsym)-based runtime, which does a bit more filtering on the expressions and also solves them using Z3.
The integration with LibAFL, however, requires you to **BYORT** (_bring your own runtime_) using the [`symcc_runtime`](https://docs.rs/symcc_runtime/0.1/symcc_runtime) crate.
This crate allows you to easily build a custom runtime out of the built-in building blocks or create entirely new runtimes with full flexibility.
Checkout out the `symcc_runtime` docs for more information on how to build your own runtime.
Check out the `symcc_runtime` docs for more information on how to build your own runtime.
### SymQEMU
@ -114,7 +115,7 @@ The `symcc_runtime` crate supports this use case and runtimes built with `symcc_
## Hybrid Fuzzing in LibAFL
The LibAFL repository contains an [example hybrid fuzzer](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/libfuzzer_stb_image_concolic).
The LibAFL repository contains an [example hybrid fuzzer](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/structure_aware/libfuzzer_stb_image_concolic).
There are three main steps involved with building a hybrid fuzzer using LibAFL:
@ -123,21 +124,23 @@ There are three main steps involved with building a hybrid fuzzer using LibAFL:
3. building the fuzzer.
Note that the order of these steps is important.
For example, we need to have runtime ready before we can do instrumentation with SymCC.
For example, we need to have a runtime ready before we can do instrumentation with SymCC.
### Building a Runtime
Building a custom runtime can be done easily using the `symcc_runtime` crate.
Note, that a custom runtime is a separate shared object file, which means that we need a separate crate for our runtime.
Check out the [example hybrid fuzzer's runtime](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/libfuzzer_stb_image_concolic/runtime) and the [`symcc_runtime` docs](https://docs.rs/symcc_runtime/0.1/symcc_runtime) for inspiration.
Check out the [example hybrid fuzzer's runtime](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/structure_aware/libfuzzer_stb_image_concolic/runtime) and the [`symcc_runtime` docs](https://docs.rs/symcc_runtime/0.1/symcc_runtime) for inspiration.
### Instrumentation
There are two main instrumentation methods to make use of concolic tracing in LibAFL:
* Using an **compile-time** instrumented target with **SymCC**.
* Using a **compile-time** instrumented target with **SymCC**.
This only works when the source is available for the target and the target is reasonably easy to build using the SymCC compiler wrapper.
* Using **SymQEMU** to dynamically instrument the target at **runtime**.
This avoids a separate instrumented target with concolic tracing instrumentation and does not require source code.
This avoids building a separate instrumented target with concolic tracing instrumentation and so does not require source code.
It should be noted, however, that the 'quality' of the generated expressions can be significantly worse and SymQEMU generally produces significantly more and significantly more convoluted expressions than SymCC.
Therefore, it is recommended to use SymCC over SymQEMU when possible.
@ -148,7 +151,7 @@ How exactly this is done does not matter.
However, the SymCC compiler needs to be made aware of the location of the runtime that it should instrument against.
This is done by setting the `SYMCC_RUNTIME_DIR` environment variable to the directory which contains the runtime (typically the `target/(debug|release)` folder of your runtime crate).
The example hybrid fuzzer instruments the target in its [`build.rs` build script](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/libfuzzer_stb_image_concolic/fuzzer/build.rs#L50).
The example hybrid fuzzer instruments the target in its [`build.rs` build script](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/structure_aware/libfuzzer_stb_image_concolic/fuzzer/build.rs#L50).
It does this by cloning and building a copy of SymCC and then using this version to instrument the target.
The [`symcc_libafl` crate](https://docs.rs/symcc_libafl) contains helper functions for cloning and building SymCC.
@ -158,27 +161,27 @@ Make sure you satisfy the [build requirements](https://github.com/eurecom-s3/sym
Build SymQEMU according to its [build instructions](https://github.com/eurecom-s3/symqemu#readme).
By default, SymQEMU looks for the runtime in a sibling directory.
Since we don't have a runtime there, we need to let it know the path to your runtime by setting`--symcc-build` argument of the `configure` script to the path of your runtime.
Since we don't have a runtime there, we need to explicitly set the`--symcc-build` argument of the `configure` script to the path of your runtime.
### Building the Fuzzer
No matter the instrumentation method, the interface between the fuzzer and the instrumented target should now be consistent.
The only difference between using SymCC and SymQEMU should be the binary that represents the target:
In the case of SymCC it will be the binary that was build with instrumentation and with SymQEMU it will be the emulator binary (eg. `x86_64-linux-user/symqemu-x86_64`), followed by your uninstrumented target binary and arguments.
In the case of SymCC it will be the binary that was build with instrumentation and with SymQEMU it will be the emulator binary (eg. `x86_64-linux-user/symqemu-x86_64`), followed by your uninstrumented target binary and its arguments.
You can use the [`CommandExecutor`](https://docs.rs/libafl/0.6.0/libafl/executors/command/struct.CommandExecutor.html) to execute your target ([example](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/libfuzzer_stb_image_concolic/fuzzer/src/main.rs#L244)).
When configuring the command, make sure you pass the `SYMCC_INPUT_FILE` environment variable the input file path, if your target reads input from a file (instead of standard input).
You can use the [`CommandExecutor`](https://docs.rs/libafl/latest/libafl/executors/command/struct.CommandExecutor.html) to execute your target ([example](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/structure_aware/libfuzzer_stb_image_concolic/fuzzer/src/main.rs#L244)).
When configuring the command, make sure you pass the `SYMCC_INPUT_FILE` environment variable (set to the input file path), if your target reads input from a file (instead of standard input).
#### Serialization and Solving
While it is perfectly possible to build a custom runtime that also performs the solving step of hybrid fuzzing in the context of the target process, the intended use of the LibAFL concolic tracing support is to serialize the (filtered and pre-processed) branch conditions using the [`TracingRuntime`](https://docs.rs/symcc_runtime/0.1/symcc_runtime/tracing/struct.TracingRuntime.html).
This serialized representation can be deserialized in the fuzzer process for solving using a [`ConcolicObserver`](https://docs.rs/libafl/0.6.0/libafl/observers/concolic/struct.ConcolicObserver.html) wrapped in a [`ConcolicTracingStage`](https://docs.rs/libafl/0.6.0/libafl/stages/concolic/struct.ConcolicTracingStage.html), which will attach a [`ConcolicMetadata`](https://docs.rs/libafl/0.6.0/libafl/observers/concolic/struct.ConcolicMetadata.html) to every [`TestCase`](https://docs.rs/libafl/0.6.0/libafl/corpus/testcase/struct.Testcase.html).
This serialized representation can be deserialized in the fuzzer process for solving using a [`ConcolicObserver`](https://docs.rs/libafl/latest/libafl/observers/concolic/struct.ConcolicObserver.html) wrapped in a [`ConcolicTracingStage`](https://docs.rs/libafl/latest/libafl/stages/concolic/struct.ConcolicTracingStage.html), which will attach a [`ConcolicMetadata`](https://docs.rs/libafl/latest/libafl/observers/concolic/struct.ConcolicMetadata.html) to every [`TestCase`](https://docs.rs/libafl/latest/libafl/corpus/testcase/struct.Testcase.html).
The `ConcolicMetadata` can be used to replay the concolic trace and solved using an SMT-Solver.
The `ConcolicMetadata` can be used to replay the concolic trace and to solve the conditions using an SMT-Solver.
Most use-cases involving concolic tracing, however, will need to define some policy around which branches they want to solve.
The [`SimpleConcolicMutationalStage`](https://docs.rs/libafl/0.6.0//libafl/stages/concolic/struct.SimpleConcolicMutationalStage.html) can be used for testing purposes.
The [`SimpleConcolicMutationalStage`](https://docs.rs/libafl/latest/libafl/stages/concolic/struct.SimpleConcolicMutationalStage.html) can be used for testing purposes.
It will attempt to solve all branches, like the original simple backend from SymCC, using Z3.
### Example
The example fuzzer shows how to use the [`ConcolicTracingStage` together with the `SimpleConcolicMutationalStage`](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/libfuzzer_stb_image_concolic/fuzzer/src/main.rs#L222) to build a basic hybrid fuzzer.
The example fuzzer shows how to use the [`ConcolicTracingStage` together with the `SimpleConcolicMutationalStage`](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/structure_aware/libfuzzer_stb_image_concolic/fuzzer/src/main.rs#L222) to build a basic hybrid fuzzer.
@ -4,7 +4,7 @@ LibAFL supports different instrumentation engines for binary-only fuzzing.
A potent cross-platform (Windows, MacOS, Android, Linux, iOS) option for binary-only fuzzing is Frida; the dynamic instrumentation tool.
In this section, we will talk about the components in fuzzing with `libafl_frida`.
You can take a look at a working example in our [`fuzzers/frida_libpng`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/frida_libpng) folder for Linux, and [`fuzzers/frida_gdiplus`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/frida_gdiplus) for Windows.
You can take a look at a working example in our [`fuzzers/binary_only/frida_libpng`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/binary_only/frida_libpng) folder for Linux, and [`fuzzers/binary_only/frida_windows_gdiplus`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/binary_only/frida_windows_gdiplus) for Windows.
## Dependencies
@ -17,7 +17,7 @@ If you are on Windows, you'll need to install llvm tools.
LibAFL uses Frida's [__Stalker__](https://frida.re/docs/stalker/) to trace the execution of your program and instrument your harness.
Thus, you have to compile your harness to a dynamic library. Frida instruments your PUT after dynamically loading it.
For example in our `frida_libpng` example, we load the dynamic library and find the symbol to harness as follows:
In our `frida_libpng` example, we load the dynamic library and find the symbol to harness as follows:
```rust,ignore
let lib = libloading::Library::new(module_name).unwrap();
@ -28,9 +28,9 @@ For example in our `frida_libpng` example, we load the dynamic library and find
## `FridaInstrumentationHelper` and Runtimes
To use functionalities that Frida offers, we'll first need to obtain `Gum` object by `Gum::obtain()`.
To use functionalities that Frida offers, we'll first need to obtain a `Gum` object by `Gum::obtain()`.
In LibAFL, we use the `FridaInstrumentationHelper` struct to manage frida-related state. `FridaInstrumentationHelper` is a key component that sets up the [__Transformer__](https://frida.re/docs/stalker/#transformer) that is used to generate the instrumented code. It also initializes the `Runtimes` that offer various instrumentation.
In LibAFL, we use the `FridaInstrumentationHelper` struct to manage frida-related state. `FridaInstrumentationHelper` is a key component that sets up the [__Transformer__](https://frida.re/docs/stalker/#transformer) that is used to generate the instrumented code. It also initializes the `Runtimes` that offer various instrumentations.
We have `CoverageRuntime` that can track the edge coverage, `AsanRuntime` for address sanitizer, `DrCovRuntime` that uses [__DrCov__](https://dynamorio.org/page_drcov.html) for coverage collection (to be imported in coverage tools like Lighthouse, bncov, dragondance,...), and `CmpLogRuntime` for cmplog instrumentation.
All of these runtimes can be slotted into `FridaInstrumentationHelper` at build time.
@ -53,12 +53,12 @@ Combined with any `Runtime` you'd like to use, you can initialize the `FridaInst
## Running the Fuzzer
After setting up the `FridaInstrumentationHelper`. You can obtain the pointer to the coverage map by calling `map_ptr_mut()`.
After setting up the `FridaInstrumentationHelper` you can obtain the pointer to the coverage map by calling `map_mut_ptr()`.
```rust,ignore
let edges_observer = HitcountsMapObserver::new(StdMapObserver::new_from_ptr(
let edges_observer = HitcountsMapObserver::new(StdMapObserver::from_mut_ptr(
"edges",
frida_helper.map_ptr_mut().unwrap(),
frida_helper.map_mut_ptr().unwrap(),
MAP_SIZE,
));
```
@ -73,7 +73,7 @@ You can then link this observer to `FridaInProcessExecutor` as follows:
tuple_list!(
edges_observer,
time_observer,
AsanErrorsObserver::new(&ASAN_ERRORS)
AsanErrorsObserver::from_static_asan_errors()
),
&mut fuzzer,
&mut state,
@ -83,5 +83,5 @@ You can then link this observer to `FridaInProcessExecutor` as follows:
);
```
And, finally you can run the fuzzer.
See the `frida_` examples in [`./fuzzers`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/) for more information and, for linux or full-system, play around with `libafl_qemu`, another binary-only tracer.
And finally you can run the fuzzer.
See the `frida_` examples in [`./fuzzers/binary_only`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/binary_only/) for more information and, for linux or full-system, play around with `libafl_qemu`, another binary_only tracer.
NYX supports both source-based and binary-only fuzzing.
Currently, `libafl_nyx` only supports [afl++](https://github.com/AFLplusplus/AFLplusplus)'s instruction. To install it, you can use `sudo apt install aflplusplus`. Or compile from the source:
Currently, `libafl_nyx` only supports [afl++](https://github.com/AFLplusplus/AFLplusplus)'s instruction type. To install it, you can use `sudo apt install aflplusplus`. Or compile from the source:
make all # this will not compile afl's additional extension
make all # this will not compile afl's additional extensions
```
Then you should compile the target with the afl++ compiler wrapper:
@ -20,11 +20,11 @@ export CXX=afl-clang-fast++
make
```
For binary-only fuzzing, Nyx uses intel-PT(Intel® Processor Trace). You can find the supported CPU at <https://www.intel.com/content/www/us/en/support/articles/000056730/processors.html>.
For binary-only fuzzing, Nyx uses intel-PT(Intel® Processor Trace). You can find the list of supported CPUs at <https://www.intel.com/content/www/us/en/support/articles/000056730/processors.html>.
## Preparing Nyx working directory
## Preparing the Nyx working directory
This step is used to pack the target into Nyx's kernel. Don't worry, we have a template shell script in our [example](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/nyx_libxml2_parallel/setup_libxml2.sh):
This step is used to pack the target into Nyx's kernel. Don't worry, we have a template shell script in our [example](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/full_system/nyx_libxml2_parallel/setup_libxml2.sh):
In the [example fuzzer](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/nyx_libxml2_standalone/src/main.rs). First you need to run `./setup_libxml2.sh`, It will prepare your target and create your nyx work directory in `/tmp/libxml2`. After that, you can start write your code.
In the [example fuzzer](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/full_system/nyx_libxml2_standalone/src/main.rs) you first need to run `./setup_libxml2.sh`. It will prepare your target and create your nyx work directory in `/tmp/libxml2`. After that, you can start to write your code.
First, to create `Nyxhelper`:
@ -57,22 +57,21 @@ First, to create `Nyxhelper`:
let share_dir = Path::new("/tmp/nyx_libxml2/");
let cpu_id = 0; // use first cpu
let parallel_mode = false; // close parallel_mode
let mut helper = NyxHelper::new(share_dir, cpu_id, true, parallel_mode, None).unwrap(); // we don't the set the last parameter in standalone mode, we just use None, here
let mut helper = NyxHelper::new(share_dir, cpu_id, true, parallel_mode, None).unwrap(); // we don't need to set the last parameter in standalone mode, we just use None, here
```
Then, fetch `trace_bits`, create an observer and the `NyxExecutor`:
```rust,ignore
let trace_bits = unsafe { std::slice::from_raw_parts_mut(helper.trace_bits, helper.map_size) };
let observer = StdMapObserver::new("trace", trace_bits);
let observer = unsafe { StdMapObserver::from_mut_ptr("trace", helper.trace_bits, helper.map_size) };
let mut executor = NyxExecutor::new(&mut helper, tuple_list!(observer)).unwrap();
```
Finally, use them as normal and pass them into `fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)` to start fuzzing.
Finally, use them normally and pass them into `fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)` to start fuzzing.
## Parallel fuzzing
In the [example fuzzer](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/nyx_libxml2_parallel/src/main.rs). First you need to run `./setup_libxml2.sh` as described before.
In the [example fuzzer](https://github.com/AFLplusplus/LibAFL/blob/main/fuzzers/full_system/nyx_libxml2_parallel/src/main.rs) you first need to run `./setup_libxml2.sh` as described before.
Parallel fuzzing relies on [`Launcher`](../message_passing/spawn_instances.md), so spawn logic should be written in the scoop of anonymous function `run_client`:
@ -91,7 +90,7 @@ let mut helper = NyxHelper::new(
cpu_id, // current cpu id
true, // open snap_mode
parallel_mode, // open parallel mode
Some(parent_cpu_id.id as u32), // the cpu-id of master instance, there is only one master instance, other instances will be treated as slaved
Some(parent_cpu_id.id as u32), // the cpu-id of main instance, there is only one main instance, other instances will be treated as secondaries
)
.unwrap();
```
@ -99,13 +98,11 @@ let mut helper = NyxHelper::new(
Then you can fetch the trace_bits and create an observer and `NyxExecutor`
In modern fuzzing situations, it is very common to have multiple fuzzing instances running. LibAFL supports monitoring mutiple fuzzing instances with `StatsdMonitor`, which can be easily enabled by adding `statsd_monitor` feature in `Cargo.toml`:
```toml
[dependencies]
libafl = { version = "*", features = ["statsd_monitor"]}
```
After setting up a StatsD monitor infrastructure, you could view all of your fuzzing instances in a browser like this one (each line represents metrics of one fuzzing instance).

## Introduction to StatsD Architecture
A typical StatsD setup involves four participants: multiple StatsD clients, a StatsD aggregator, a Prometheus, and a visualizer.
Normally, each fuzzing instance is bound to a StatsD client (the `StatsdMonitor`). The StatsD client pushes fuzzing metrics (e.g. exec per second) towards the StatsD aggregator consistently. The Prometheus server is the center of StatsD metrics, which polls StatsD metrics from the StatsD aggregator at a specific polling frequency. The visualizer queries the Prometheus server, and display the metrics for further analysis and monitoring.
## Set up a StatsD Monitor Infrastructure
The easiest way to install and set up the infrastructure is with Docker and Docker Compose. To begin with, create a directory with the following files:
* `docker-compose.yml`
```yml
networks:
statsd-net:
driver: bridge
volumes:
grafana-data:
services:
prometheus:
image: prom/prometheus
container_name: prometheus
volumes:
- ./prometheus.yml:/prometheus.yml
command:
- '--config.file=/prometheus.yml'
restart: unless-stopped
ports:
- "9090:9090"
networks:
- statsd-net
statsd_exporter:
image: prom/statsd-exporter
container_name: statsd_exporter
volumes:
- ./statsd_mapping.yml:/statsd_mapping.yml
command:
- "--statsd.mapping-config=/statsd_mapping.yml"
ports:
- "9102:9102/tcp"
- "8125:9125/udp"
networks:
- statsd-net
grafana:
image: grafana/grafana
container_name: grafana
volumes:
- grafana-data:/var/lib/grafana
restart: unless-stopped
ports:
- "3000:3000"
networks:
- statsd-net
```
* `prometheus.yml`
```yml
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'fuzzing_metrics'
static_configs:
- targets: ['statsd_exporter:9102']
```
* `statsd_mapping.yml`
```yml
mappings:
- match: "fuzzing.*"
name: "fuzzing"
labels:
type: "$1"
```
And use
```shell
docker compose up -d
```
Then the basic infrastructure is set up.
It is worth noting that all participants are not necessarily be in the same machine, as long as they could communicate with each other with network. Specifically, the StatsD aggregator should be accessible from StatsD clients and Prometheus server, and the Prometheus server should be accessible from the visualizer.
In this infrastructure setup, we choose the [prometheus/statsd-exporter](https://github.com/prometheus/statsd_exporter) as a StatsD aggregator, which aggregates metrics from multiple StatsD clients, and converts the metrics into Prometheus format. And we choose the [Grafana](https://grafana.com) as the visualizer.
### Configure Grafana
After the infrastructure is set up, you should use a browser and go to `localhost:3000` to configure Grafana, the visualizer, and view StatsD metrics (the default username and password for Grafana docker login is admin/admin).
First, you should add the Prometheus server as data source for Grafana, as described in [Add a data source](https://grafana.com/docs/grafana/latest/datasources/#add-a-data-source). By picking "Prometheus" as datasource type, you could create a data source with name "prometheus", and use `http://prometheus:9090` as Prometheus server URL (Docker network could automatically resolve the host name to the IP of prometheus service, and the port is specified at `docker-compose.yml` above).
Then, you should create a dashboard to view the metrics. You could simply importing [this dashboard template](../resources/grafana-libafl.json), or create a dashboard manually by following [this official guide](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/create-dashboard/#create-a-dashboard).
After the dashboard is created, all metrics can be viewed directly.
## Use `StatsdMonitor` in LibAFL
It is rather easy to use the `StatsdMonitor` in LibAFL. Let's say you were using a `MultiMonitor` previously:
```rust,ignore
let monitor = MultiMonitor::new(|s| println!("{s}"));
```
Then you could just create a `StatsdMonitor` and combine this two monitors with a `tuple_list!`:
```rust,ignore
let multi_monitor = MultiMonitor::new(|s| println!("{s}"));
let statsd_monitor = StatsdMonitor::new("localhost".to_string(), 8125, StatsdMonitorTagFlavor::default());
let monitor = tuple_list!(multi_monitor, statsd_monitor);
```
Then the monitor will automatically push StatsD metrics towards the StatsD aggregator at localhost:8125, which is specified in the `docker-compose.yml` above.
### Tag Flavor
The vanilla StatsD metrics do not have a mechanism to distinguish different StatsD clients, which is inconvenience to view multiple fuzzing instances. Several StatsD servers have introduced a new mechanism called tag to solve this problem, and in LibAFL, we currently support [Datadog](https://docs.datadoghq.com/developers/dogstatsd/) style tag, which is the default for `StatsdMonitorTagFlavor`.
To use tag correctly, we should create a `StatsdMonitorTagFlavor` with a tag identifier and optional custom tags.
```rust,ignore
let tag_flavor = StatsdMonitorTagFlavor::DogStatsd {
let statsd_monitor = StatsdMonitor::new("localhost".to_string(), 8125, tag_flavor);
```
The tag identifier is used to differentiate StatsD clients, which should be unique for each fuzzing instances. The optional custom tags could be used by Grafana users to filter clients based on custom informations.
## FAQ
### What can I do if I cannot see any metrics in Grafana?
To debug the `StatsdMonitor`, you could first comment out all other monitors for clarity. The you could setup a log implementation such as [`env_logger`](https://crates.io/crates/env_logger) at the top of your `main` function. For `env_logger`, run the target application with environment `RUST_LOG=info`, and you could see logs from LibAFL. The output logs should contain error logs of `StatsdMonitor` if connection is down or anything unexpected.
If you still have no idea why things went wrong, please [file an issue](https://github.com/AFLplusplus/LibAFL/issues/new/choose) to LibAFL.
### Does network interactions of StatsD decrease the fuzzing throughput?
Thanks to the [`cadence`](https://crates.io/crates/cadence) crate which we used as StatsD client implementation, the network interaction is conduct in an individual thread, so the affect from network interactions could be negligible.
### Can I optionally enable the `StatsdMonitor` via something like commandline options?
You can wrap the `StatsdMonitor` with an `OptionalMonitor`:
```rust,ignore
let statsd_monitor = if commandline_is_set {
Some(StatsdMonitor::new(/* ... */))
} else {
None
};
let optional_statsd_monitor = OptionalMonitor::new(statsd_monitor);
```
### What is the different between `StatsdMonitor` and `PrometheusMonitor`?
The `PrometheusMonitor` could be used to view detailed statistics about one fuzzing instance (with multiple clients maybe), since it can show the statistics about individual clients. And `PrometheusMonitor` is implemented as a Prometheus datasource, which will, based on the pull-architecture of Prometheus, occupy a port for each instance for Prometheus server to access.
The `StatsdMonitor` is used to view aggregated statistics about multiple fuzzing instances (showing individual client statistics will make the visualized graph very distracting). And due to the push-architecture of StatsD server, we only need one port for the StatsD aggregator to receive metrics from each client.
## References
* [AFL++'s docs about rpc_statsd](https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/rpc_statsd.md)
@ -6,7 +6,7 @@ While the following chapters discuss the components of LibAFL in detail, here we
We are going to fuzz a simple Rust function that panics under a condition. The fuzzer will be single-threaded and will stop after the crash, just like libFuzzer normally does.
You can find a complete version of this tutorial as an example fuzzer in [`fuzzers/baby_fuzzer`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/baby_fuzzer).
You can find a complete version of this tutorial as an example fuzzer in [`fuzzers/baby/baby_fuzzer`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/baby/baby_fuzzer).
> ### Warning
>
@ -17,7 +17,7 @@ You can find a complete version of this tutorial as an example fuzzer in [`fuzze
We use cargo to create a new Rust project with LibAFL as a dependency.
```sh
```console
$ cargo new baby_fuzzer
$ cd baby_fuzzer
```
@ -25,18 +25,11 @@ $ cd baby_fuzzer
The generated `Cargo.toml` looks like the following:
```toml
[package]
name = "baby_fuzzer"
version = "0.1.0"
authors = ["Your Name <you@example.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
In order to use LibAFl we must add it as dependency adding `libafl = { path = "path/to/libafl/" }` under `[dependencies]`.
That path actually needs to point to the `libafl` directory within the cloned repo, not the root of the repo itself.
You can use the LibAFL version from [crates.io](https://crates.io/crates/libafl) if you want, in this case, you have to use `libafl = "*"` to get the latest version (or set it to the current version).
As we are going to fuzz Rust code, we want that a panic does not simply cause the program to exit, but raise an `abort` that can then be caught by the fuzzer.
@ -47,28 +40,10 @@ Alongside this setting, we add some optimization flags for the compilation, when
The final `Cargo.toml` should look similar to the following:
```toml
[package]
name = "baby_fuzzer"
version = "0.1.0"
authors = ["Your Name <you@example.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
- The first parameter is a random number generator, that is part of the fuzzer state, in this case, we use the default one `StdRand`, but you can choose a different one. We seed it with the current nanoseconds.
@ -129,43 +84,26 @@ let mut state = StdState::new(
To avoid type annotation error, you can use `InMemoryCorpus::<BytesInput>::new()` to replace `InMemoryCorpus::new()`. If not, type annotation will be automatically inferred when adding `executor`.
- third parameter is another corpus that stores the "solution" testcases for the fuzzer. For our purpose, the solution is the input that triggers the panic. In this case, we want to store it to disk under the `crashes` directory, so we can inspect it.
- last two parameters are feedback and objective, we will discuss them later.
- The third parameter is another Corpus that stores the "solution" testcases for the fuzzer. For our purpose, the solution is the input that triggers the panic. In this case, we want to store it to disk under the `crashes` directory, so we can inspect it.
- The last two parameters are feedback and objective, we will discuss them later.
Another required component is the **EventManager**. It handles some events such as the addition of a testcase to the corpus during the fuzzing process. For our purpose, we use the simplest one that just displays the information about these events to the user using a `Monitor` instance.
```rust,ignore
// The Monitor trait defines how the fuzzer stats are displayed to the user
let mon = SimpleMonitor::new(|s| println!("{}", s));
// The event manager handle the various events generated during the fuzzing loop
// such as the notification of the addition of a new item to the corpus
In addition, we have the **Fuzzer**, an entity that contains some actions that alter the State. One of these actions is the scheduling of the testcases to the fuzzer using a **Scheduler**.
We create it as `QueueScheduler`, a scheduler that serves testcases to the fuzzer in a FIFO fashion.
```rust,ignore
// A queue policy to get testcasess from the corpus
let scheduler = QueueScheduler::new();
// A fuzzer with feedbacks and a corpus scheduler
let mut fuzzer = StdFuzzer::new(scheduler, (), ());
Last but not least, we need an **Executor** that is the entity responsible to run our program under test. In this example, we want to run the harness function in-process (without forking off a child, for example), and so we use the `InProcessExecutor`.
When running, you should see something similar to:
```sh
```console
$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Running `target/debug/baby_fuzzer`
@ -225,60 +141,22 @@ Now we want to turn our simple fuzzer into a feedback-based one and increase the
**Observer** can record the information about properties of a fuzzing run and then feeds the fuzzer. We use the `StdMapObserver`, the default observer that uses a map to keep track of covered elements. In our fuzzer, each condition is mapped to an entry of such map.
We represent such map as a `static mut` variable.
As we don't rely on any instrumentation engine, we have to manually track the satisfied conditions by `singals_set` in our harness:
As we don't rely on any instrumentation engine, we have to manually track the satisfied conditions by `signals_set` in our harness:
```rust
extern crate libafl;
use libafl::{
bolts::AsSlice,
inputs::{BytesInput, HasTargetBytes},
executors::ExitKind,
};
// Coverage map with explicit assignments due to the lack of instrumentation
The observers are usually kept in the corresponding executor as they keep track of information that is valid for just one run. We have then to modify our InProcessExecutor creation to include the observer as follows:
```rust,ignore
// Create the executor for an in-process function with just one observer
Now that the fuzzer can observe which condition is satisfied, we need a way to rate an input as interesting (i.e. worth of addition to the corpus) based on this observation. Here comes the notion of Feedback.
@ -287,49 +165,23 @@ Now that the fuzzer can observe which condition is satisfied, we need a way to r
We use `MaxMapFeedback`, a feedback that implements a novelty search over the map of the MapObserver. Basically, if there is a value in the observer's map that is greater than the maximum value registered so far for the same entry, it rates the input as interesting and updates its state.
**Objective Feedback** is another kind of Feedback which decide if an input is a "solution". It will save input to solutions(`./crashes` in our case) other than corpus when the input is rated interesting. We use `CrashFeedback` to tell the fuzzer that if an input causes the program to crash it is a solution for us.
**Objective Feedback** is another kind of Feedback which decides if an input is a "solution". It will save input to solutions(`./crashes` in our case) rather than corpus when the input is rated interesting. We use `CrashFeedback` to tell the fuzzer that if an input causes the program to crash it is a solution for us.
We need to update our State creation including the feedback state and the Fuzzer including the feedback and the objective:
Now, after including the correct `use`, we can run the program, but the outcome is not so different from the previous one as the random generator does not take into account what we save as interesting in the corpus. To do that, we need to plug a Mutator.
Now, we can run the program, but the outcome is not so different from the previous one as the random generator does not take into account what we save as interesting in the corpus. To do that, we need to plug a Mutator.
**Stages** perform actions on individual inputs, taken from the corpus.
For instance, the `MutationalStage` executes the harness several times in a row, every time with mutated inputs.
@ -337,28 +189,20 @@ For instance, the `MutationalStage` executes the harness several times in a row,
As the last step, we create a MutationalStage that uses a mutator inspired by the havoc mutator of AFL.
Finished dev [unoptimized + debuginfo] target(s) in 1.56s
@ -378,4 +222,4 @@ Bye!
As you can see, after the panic message, the `objectives` count of the log increased by one and you will find the crashing input in `crashes/`.
The complete code can be found in [`./fuzzers/baby_fuzzer`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/baby_fuzzer) alongside other `baby_` fuzzers.
The complete code can be found in [`./fuzzers/baby/baby_fuzzer`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/baby/baby_fuzzer) alongside other `baby_` fuzzers.
@ -4,8 +4,8 @@ The Corpus is where testcases are stored. We define a Testcase as an Input and a
A Corpus can store testcases in different ways, for example on disk, or in memory, or implement a cache to speedup on disk storage.
Usually, a testcase is added to the Corpus when it is considered as interesting, but a Corpus is used also to store testcases that fulfill an objective (like crashing the tested program for instance).
Usually, a testcase is added to the Corpus when it is considered as interesting, but a Corpus is used also to store testcases that fulfill an objective (like crashing the program under test for instance).
Related to the Corpus, there is the way in which the fuzzer should ask for the next testcase to fuzz picking it from the Corpus. The taxonomy for this in LibAFL is CorpusScheduler, the entity representing the policy to pop testcases from the Corpus, FIFO for instance.
Related to the Corpus is the way in which the next testcase (the fuzzer would ask for) is retrieved from the Corpus. The taxonomy for this handling in LibAFL is Scheduler, the entity representing the policy to pop testcases from the Corpus, in a FIFO fashion for instance.
Speaking about the code, [`Corpus`](https://docs.rs/libafl/0/libafl/corpus/trait.Corpus.html) and [`CorpusScheduler`](https://docs.rs/libafl/0/libafl/corpus/trait.CorpusScheduler.html) are traits.
Speaking about the code, [`Corpus`](https://docs.rs/libafl/latest/libafl/corpus/trait.Corpus.html) and [`Scheduler`](https://docs.rs/libafl/latest/libafl/schedulers/trait.Scheduler.html) are traits.
In different fuzzers, this concept of executing the program under test means each run is now always the same.
For instance, for in-memory fuzzers like libFuzzer an execution is a call to an harness function, for hypervisor-based fuzzers like [kAFL](https://github.com/IntelLabs/kAFL) instead an entire operating system is started from a snapshot for each run.
For instance, for in-process fuzzers like libFuzzer an execution is a call to an harness function, for hypervisor-based fuzzers like [kAFL](https://github.com/IntelLabs/kAFL) instead an entire operating system is started from a snapshot for each run.
In our model, an Executor is the entity that defines not only how to execute the target, but all the volatile operations that are related to just a single run of the target.
@ -9,22 +9,22 @@ So the Executor is for instance responsible to inform the program about the inpu
In our model, it can also hold a set of Observers connected with each execution.
In Rust, we bind this concept to the [`Executor`](https://docs.rs/libafl/0/libafl/executors/trait.Executor.html) trait. A structure implementing this trait must implement [`HasObservers`](https://docs.rs/libafl/0/libafl/executors/trait.HasObservers.html) too if wants to hold a set of Observers.
In Rust, we bind this concept to the [`Executor`](https://docs.rs/libafl/latest/libafl/executors/trait.Executor.html) trait. A structure implementing this trait must implement [`HasObservers`](https://docs.rs/libafl/latest/libafl/executors/trait.HasObservers.html) too if wants to hold a set of Observers.
By default, we implement some commonly used Executors such as [`InProcessExecutor`](https://docs.rs/libafl/0/libafl/executors/inprocess/struct.InProcessExecutor.html) in which the target is a harness function providing in-process crash detection. Another Executor is the [`ForkserverExecutor`](https://docs.rs/libafl/0/libafl/executors/forkserver/struct.ForkserverExecutor.html) that implements an AFL-like mechanism to spawn child processes to fuzz.
A common pattern when creating an Executor is wrapping an existing one, for instance [`TimeoutExecutor`](https://docs.rs/libafl/0.6.1/libafl/executors/timeout/struct.TimeoutExecutor.html) wraps an executor and install a timeout callback before calling the original run function of the wrapped executor.
By default, we implement some commonly used Executors such as [`InProcessExecutor`](https://docs.rs/libafl/latest/libafl/executors/inprocess/type.InProcessExecutor.html) in which the target is a harness function providing in-process crash detection. Another Executor is the [`ForkserverExecutor`](https://docs.rs/libafl/latest/libafl/executors/forkserver/struct.ForkserverExecutor.html) that implements an AFL-like mechanism to spawn child processes to fuzz.
## InProcessExecutor
Let's begin with the base case; `InProcessExecutor`.
This executor executes the harness program (function) inside the fuzzer process.
When you want to execute the harness as fast as possible, you will most probably want to use this `InprocessExecutor`.
One thing to note here is, when your harness is likely to have heap corruption bugs, you want to use another allocator so that corrupted heap does not affect the fuzzer itself. (For example, we adopt MiMalloc in some of our fuzzers.). Alternatively you can compile your harness with address sanitizer to make sure you can catch these heap bugs.
One thing to note here is, when your harness is likely to have heap corruption bugs, you want to use another allocator so that corrupted heap does not affect the fuzzer itself. (For example, we adopt MiMalloc in some of our fuzzers.). Alternatively you can compile your harness with address sanitizer to make sure you can catch these heap bugs.
## ForkserverExecutor
Next, we'll take a look at the `ForkserverExecutor`. In this case, it is `afl-cc` (from AFLplusplus/AFLplusplus) that compiles the harness code, and therefore, we can't use `EDGES_MAP` anymore. Hopefully, we have [_a way_](https://github.com/AFLplusplus/AFLplusplus/blob/2e15661f184c77ac1fbb6f868c894e946cbb7f17/instrumentation/afl-compiler-rt.o.c#L270) to tell the forkserver which map to record the coverage.
Next, we'll take a look at the `ForkserverExecutor`. In this case, it is `afl-cc` (from AFL/AFLplusplus) that compiles the harness code, and therefore, we can't use `EDGES_MAP` anymore. Fortunately we have [_a way_](https://github.com/AFLplusplus/AFLplusplus/blob/2e15661f184c77ac1fbb6f868c894e946cbb7f17/instrumentation/afl-compiler-rt.o.c#L270) to tell the forkserver which map to record the coverage in.
As you can see from the forkserver example,
@ -32,13 +32,15 @@ As you can see from the forkserver example,
//Coverage map shared between observer and executor
let mut shmem = StdShMemProvider::new().unwrap().new_shmem(MAP_SIZE).unwrap();
//let the forkserver know the shmid
shmem.write_to_env("__AFL_SHM_ID").unwrap();
let mut shmem_buf = shmem.as_mut_slice();
unsafe {
shmem.write_to_env("__AFL_SHM_ID").unwrap();
}
let mut shmem_buf = shmem.as_slice_mut();
```
Here we make a shared memory region; `shmem`, and write this to environmental variable `__AFL_SHM_ID`. Then the instrumented binary, or the forkserver, finds this shared memory region (from the aforementioned env var) to record its coverage. On your fuzzer side, you can pass this shmem map to your `Observer` to obtain coverage feedbacks combined with any `Feedback`.
Another feature of the `ForkserverExecutor` to mention is the shared memory testcases. In normal cases, the mutated input is passed between the forkserver and the instrumented binary via `.cur_input` file. You can improve your forkserver fuzzer's performance by passing the input with shared memory.
Another feature of the `ForkserverExecutor` to mention is the shared memory testcases. In normal cases, the mutated input is passed between the forkserver and the instrumented binary via `.cur_input` file. You can improve your forkserver fuzzer's performance by passing the input with shared memory.
If the target is configured to use shared memory testcases, the `ForkserverExecutor` will notice this during the handshake and will automatically set up things accordingly.
See AFL++'s [_documentation_](https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.persistent_mode.md#5-shared-memory-fuzzing) or the fuzzer example in `forkserver_simple/src/program.c` for reference.
@ -48,7 +50,7 @@ See AFL++'s [_documentation_](https://github.com/AFLplusplus/AFLplusplus/blob/st
Finally, we'll talk about the `InProcessForkExecutor`.
`InProcessForkExecutor` has only one difference from `InprocessExecutor`; It forks before running the harness and that's it.
But why do we want to do so? well, under some circumstances, you may find your harness pretty unstable or your harness wreaks havoc on the global states. In this case, you want to fork it before executing the harness runs in the child process so that it doesn't break things.
But why do we want to do so? Well, under some circumstances, you may find your harness pretty unstable or your harness wreaks havoc on the global states. In this case, you want to fork it before executing the harness runs in the child process so that it doesn't break things.
However, we have to take care of the shared memory, it's the child process that runs the harness code and writes the coverage to the map.
@ -59,12 +61,25 @@ On your fuzzer side, you can allocate a shared memory region and make the `EDGES
@ -10,17 +10,25 @@ The concept of "interestingness" is abstract, but typically it is related to a n
As an example, given an Observer that reports all the sizes of memory allocations, a maximization Feedback can be used to maximize these sizes to sport pathological inputs in terms of memory consumption.
In terms of code, the library offers the [`Feedback`](https://docs.rs/libafl/0/libafl/feedbacks/trait.Feedback.html) and the [`FeedbackState`](https://docs.rs/libafl/0/libafl/feedbacks/trait.FeedbackState.html) traits.
The first is used to implement functors that, given the state of the observers from the last execution, tells if the execution was interesting. The second is tied with `Feedback` and it is the state of the data that the feedback wants to persist in the fuzzers's state, for instance the cumulative map holding all the edges seen so far in the case of a feedback based on edge coverage.
In terms of code, the library offers the [`Feedback`](https://docs.rs/libafl/latest/libafl/feedbacks/trait.Feedback.html) trait.
It is used to implement functors that, given the state of the observers from the last execution, tells if the execution was interesting.
So to speak, it reduces the observations to a boolean result of [`is_interesting`](https://docs.rs/libafl/latest/libafl/feedbacks/trait.Feedback.html#tymethod.is_interesting) - or not.
For this, a `Feedback` can store anything it wants to persist in the fuzzers's state.
This might be, for instance, the cumulative map of all edges seen so far, in the case of a feedback based on edge coverage.
This can be achieved by adding `Metadata` in [`init_state`](https://docs.rs/libafl/latest/libafl/feedbacks/trait.Feedback.html#method.init_state) and accessing it later in `is_interesting`.
`Feedback` can also add custom metadata to a newly created [`Testcase`](https://docs.rs/libafl/latest/libafl/corpus/testcase/struct.Testcase.html) using [`append_metadata`](https://docs.rs/libafl/latest/libafl/feedbacks/trait.Feedback.html#method.append_metadata).
Multiple Feedbacks can be combined into boolean formula, considering for instance an execution as interesting if it triggers new code paths or execute in less time compared to the average execution time using [`feedback_or`](https://docs.rs/libafl/*/libafl/macro.feedback_or.html).
Multiple Feedbacks can be combined into a boolean expression, considering for instance an execution as interesting if it triggers new code paths or execute in less time compared to the average execution time using [`feedback_or`](https://docs.rs/libafl/latest/libafl/macro.feedback_or.html).
On top, logic operators like `feedback_or` and `feedback_and` have a `_fast` option (`feedback_or_fast` where the second feedback will not be evaluated, if the first part already answers the `interestingness` question, to save precious performance.
On top, logic operators like `feedback_or` and `feedback_and` have a `_fast`variant (e.g. `feedback_or_fast`) where the second feedback will not be evaluated, if the value of the first feedback operand already answers the `interestingness` question so as to save precious performance.
Using `feedback_and_fast` in combination with [`ConstFeedback`](https://docs.rs/libafl/*/libafl/feedbacks/enum.ConstFeedback.html#method.new), certain feedbacks can be disabled dynamically.
Using `feedback_and_fast` in combination with [`ConstFeedback`](https://docs.rs/libafl/latest/libafl/feedbacks/enum.ConstFeedback.html#method.new), certain feedbacks can be disabled dynamically.
## Objectives
While feedbacks are commonly used to decide if an [`Input`](https://docs.rs/libafl/*/libafl/inputs/trait.Input.html) should be kept for future mutations, they serve a double-purpose, as so-called `Objective Feedbacks`.
In this case, the `interestingness` of a feedback indicates, if an `Objective` has been hit.
Commonly, these would be a`crash or a timeout, but they can also be used to find specific parts of the program, for sanitization, or a differential fuzzing success.
While feedbacks are commonly used to decide if an [`Input`](https://docs.rs/libafl/latest/libafl/inputs/trait.Input.html) should be kept for future mutations, they serve a double-purpose, as so-called `Objective Feedbacks`.
In this case, the `interestingness` of a feedback indicates if an `Objective` has been hit.
Commonly, these objectives would be a crash or a timeout, but they can also be used to detect if specific parts of the program have been reached, for sanitization, or a differential fuzzing success.
Objectives use the same trait as a normal [`Feedback`](https://docs.rs/libafl/latest/libafl/feedbacks/trait.Feedback.html) and the implementations can be used interchangeably.
The only difference is that `interesting` Objectives won't be mutated further, and are counted as `Solutions`, a successful fuzzing campaign.
@ -6,4 +6,4 @@ Typically, a random generator is used to generate random inputs.
Generators are traditionally less used in Feedback-driven Fuzzing, but there are exceptions, like Nautilus, that uses a Grammar generator to create the initial corpus and a sub-tree Generator as a mutation of its grammar Mutator.
In the code, [`Generator`](https://docs.rs/libafl/0/libafl/generators/trait.Generator.html) is a trait.
In the code, [`Generator`](https://docs.rs/libafl/latest/libafl/generators/trait.Generator.html) is a trait.
@ -6,10 +6,10 @@ In our model of an abstract fuzzer, we define the Input as the internal represen
In the straightforward case, the input of the program is a byte array and in fuzzers such as AFL we store and manipulate exactly these byte arrays.
But it is not always the case. A program can expect inputs that are not byte arrays (e.g. a sequence of syscalls) and the fuzzer does not represent the Input in the same way that the program consumes it.
But it is not always the case. A program can expect inputs that are not linear byte arrays (e.g. a sequence of syscalls forming a use case or protocol) and the fuzzer does not represent the Input in the same way that the program consumes it.
In case of a grammar fuzzer for instance, the Input is generally an Abstract Syntax Tree because it is a data structure that can be easily manipulated while maintaining the validity, but the program expects a byte array as input, so just before the execution, the tree is serialized to a sequence of bytes.
In the Rust code, an [`Input`](https://docs.rs/libafl/*/libafl/inputs/trait.Input.html) is a trait that can be implemented only by structures that are serializable and have only owned data as fields.
In the Rust code, an [`Input`](https://docs.rs/libafl/latest/libafl/inputs/trait.Input.html) is a trait that can be implemented only by structures that are serializable and have only owned data as fields.
While most fuzzer use a normal `BytesInput`], more advanced inputs like inputs include special inputs for grammar fuzzing ([GramatronInput](https://docs.rs/libafl/*/libafl/inputs/gramatron/struct.GramatronInput.html) or `NautilusInput` on nightly), as well as the token-level [EncodedInput](https://docs.rs/libafl/*/libafl/inputs/encoded/struct.EncodedInput.html).
While most fuzzers use a normal `BytesInput`, more advanced ones use inputs that include special inputs for grammar fuzzing ([GramatronInput](https://docs.rs/libafl/latest/libafl/inputs/gramatron/struct.GramatronInput.html) or `NautilusInput` on Rust nightly), as well as the token-level [EncodedInput](https://docs.rs/libafl/latest/libafl/inputs/encoded/struct.EncodedInput.html).
The Mutator is an entity that takes one or more Inputs and generates a new derived one.
The Mutator is an entity that takes one or more Inputs and generates a new instance of Input derived by its inputs.
Mutators can be composed, and they are generally linked to a specific Input type.
There can be, for instance, a Mutator that applies more than a single type of mutation on the input. Consider a generic Mutator for a byte stream, bit flip is just one of the possible mutations but not the only one, there is also, for instance, the random replacement of a byte of the copy of a chunk.
There can be, for instance, a Mutator that applies more than a single type of mutation to the input. Consider a generic Mutator for a byte stream, bit flip is just one of the possible mutations but not the only one, there is also, for instance, the random replacement of a byte of the copy of a chunk.
In LibAFL, [`Mutator`](https://docs.rs/libafl/*/libafl/mutators/trait.Mutator.html) is a trait.
There are also mutators that always produce valid inputs, say a mutator that generates valid JSON or code, but these grammar based mutators need a grammar to work.
In LibAFL, [`Mutator`](https://docs.rs/libafl/latest/libafl/mutators/trait.Mutator.html) is a trait.
@ -4,10 +4,10 @@ An Observer is an entity that provides an information observed during the execut
The information contained in the Observer is not preserved across executions, but it may be serialized and passed on to other nodes if an `Input` is considered `interesting`, and added to the `Corpus`.
As an example, the coverage map, filled during the execution to report the executed edges used by fuzzers such as AFL and `HonggFuzz` can be considered an observation. Another `Observer` can be the time spent executing a run, the program output, or more advanced observation, like maximum stack depth at runtime.
This information is not preserved across runs, and it is an observation of a dynamic property of the program.
As an example, the coverage map, filled during the execution to report the executed edges used by fuzzers such as AFL and `HonggFuzz` can be considered an observation. Another `Observer` can collect the time spent executing a run, the program output, or a more advanced observation, like maximum stack depth at runtime.
This information is an observation of a dynamic property of the program.
In terms of code, in the library this entity is described by the [`Observer`](https://docs.rs/libafl/0/libafl/observers/trait.Observer.html) trait.
In terms of code, in the library this entity is described by the [`Observer`](https://docs.rs/libafl/latest/libafl/observers/trait.Observer.html) trait.
In addition to holding the volatile data connected with the last execution of the target, the structures implementing this trait can define some execution hooks that are executed before and after each fuzz case. In these hooks, the observer can modify the fuzzer's state.
A Stage is an entity that operates on a single Input got from the Corpus.
A Stage is an entity that operates on a single Input received from the Corpus.
For instance, a Mutational Stage, given an input of the corpus, applies a Mutator and executes the generated input one or more time. How many times this has to be done can be scheduled, AFL for instance uses a performance score of the input to choose how many times the havoc mutator should be invoked. This can depend also on other parameters, for instance, the length of the input if we want to just apply a sequential bitflip, or be a fixed value.
For instance, a Mutational Stage, given an input of the corpus, applies a Mutator and executes the generated input one or more times. How many times this has to be done can be scheduled, AFL for instance uses a performance score of the input to choose how many times the havoc mutator should be invoked. This can depend also on other parameters, for instance, the length of the input if we want to just apply a sequential bitflip, or a fixed value.
A stage can also be an analysis stage, for instance, the Colorization stage of Redqueen that aims to introduce more entropy in a testcase or the Trimming stage of AFL that aims to reduce the size of a testcase.
There are several stages in the LibAFL codebase implementing the [`Stage`](https://docs.rs/libafl/*/libafl/stages/trait.Stage.html) trait.
There are several stages in the LibAFL codebase implementing the [`Stage`](https://docs.rs/libafl/latest/libafl/stages/trait.Stage.html) trait.
@ -8,8 +8,8 @@ The LibAFL code reuse mechanism is based on components, rather than sub-classes,
Thinking about similar fuzzers, you can observe that most of the time the data structures that are modified are the ones related to testcases and the fuzzer global state.
Beside the entities previously described, we introduce the [`Testcase`](https://docs.rs/libafl/0.6/libafl/corpus/testcase/struct.Testcase.html) and [`State`](https://docs.rs/libafl/0.6/libafl/state/struct.StdState.html) entities. The Testcase is a container for an Input stored in the Corpus and its metadata (so, in the implementation, the Corpus stores Testcases) and the State contains all the metadata that are evolved while running the fuzzer, Corpus included.
Beside the entities previously described, we introduce the [`Testcase`](https://docs.rs/libafl/latest/libafl/corpus/testcase/struct.Testcase.html) and [`State`](https://docs.rs/libafl/latest/libafl/state/struct.StdState.html) entities. The Testcase is a container for an Input stored in the Corpus and its metadata (so, in the implementation, the Corpus stores Testcases) and the State contains all the metadata that are evolved while running the fuzzer, Corpus included.
The State, in the implementation, contains only owned objects that are serializable, and it is serializable itself. Some fuzzers may want to serialize its state when pausing or just, when doing in-process fuzzing, serialize on crash and deserialize in the new process to continue to fuzz with all the metadata preserved.
The State, in the implementation, contains only owned objects that are serializable, and it is serializable itself. Some fuzzers may want to serialize their state when pausing or just, when doing in-process fuzzing, serialize on crash and deserialize in the new process to continue to fuzz with all the metadata preserved.
Additionally, we group the entities that are "actions", like the `CorpusScheduler` and the `Feedbacks`, in a common place, the [`Fuzzer'](https://docs.rs/libafl/*/libafl/fuzzer/struct.StdFuzzer.html).
Additionally, we group the entities that are "actions", like the `CorpusScheduler` and the `Feedbacks`, in a common place, the [`Fuzzer`](https://docs.rs/libafl/latest/libafl/fuzzer/struct.StdFuzzer.html).
The struct must be static, so it cannot hold references to borrowed objects.
As an alternative to `derive(SerdeAny)`that is a proc-macro in `libafl_derive` the user can use `libafl::impl_serdeany!(MyMetadata);`.
As an alternative to `derive(SerdeAny)`which is a proc-macro in `libafl_derive` the user can use `libafl_bolts::impl_serdeany!(MyMetadata);`.
## Usage
Metadata objects are primarly intended to be used inside [`SerdeAnyMap`](https://docs.rs/libafl/0.5.0/libafl/bolts/serdeany/serdeany_registry/struct.SerdeAnyMap.html) and [`NamedSerdeAnyMap`](https://docs.rs/libafl/0.5.0/libafl/bolts/serdeany/serdeany_registry/struct.NamedSerdeAnyMap.html).
Metadata objects are primarly intended to be used inside [`SerdeAnyMap`](https://docs.rs/libafl_bolts/latest/libafl_bolts/serdeany/serdeany_registry/struct.SerdeAnyMap.html) and [`NamedSerdeAnyMap`](https://docs.rs/libafl_bolts/latest/libafl_bolts/serdeany/serdeany_registry/struct.NamedSerdeAnyMap.html).
With these maps, the user can retrieve instances by type (and name). Internally, the instances are stored as SerdeAny trait objects.
Structs that want to have a set of metadata must implement the [`HasMetadata`](https://docs.rs/libafl/0.5.0/libafl/state/trait.HasMetadata.html) trait.
Structs that want to have a set of metadata must implement the [`HasMetadata`](https://docs.rs/libafl/latest/libafl/common/trait.HasMetadata.html) trait.
By default, Testcase and State implement it and hold a SerdeAnyMap testcase.
We moved the old `libafl::bolts` module to its own crate called `libafl_bolts`.
For this, imports for types in LibAFL bolts have changed in version 0.11, everything else should remain the same.
## Reasons for This Change
With the change we can now use a lot of low-level features of LibAFL for projects that are unrelated to fuzzing, or just completely different to LibAFL.
Some cross-platform things in bolts include
* SerdeAnyMap: a map that stores and retrieves elements by type and is serializable and deserializable
* LLMP: A fast, lock-free IPC mechanism via SharedMap
* Core_affinity: A maintained version of `core_affinity` that can be used to get core information and bind processes to cores
* Rands: Fast random number generators for fuzzing (like [RomuRand](https://www.romu-random.org/))
* MiniBSOD: get and print information about the current process state including important registers.
* Tuples: Haskel-like compile-time tuple lists
* Os: OS specific stuff like signal handling, windows exception handling, pipes, and helpers for `fork`
## What changed
You will need to move all `libafl::bolts::` imports to `libafl_bolts:::` and add the crate dependency in your Cargo.toml (and specify feature flags there).
As only exception, the `libafl::bolts::launcher::Launcher` has moved to `libafl::events::launcher::Launcher` since it has fuzzer and `EventManager` specific code.
If you are using `prelude`, you may need to also ad `libafl_bolts::prelude`.
We deleted `TimeoutExecutor` and `TimeoutForkserverExecutor` and make it mandatory for `InProcessExecutor` and `ForkserverExecutor` to have the timeout. Now `InProcessExecutor` and `ForkserverExecutor` have the default timeout of 5 seconds.
## Reason for This Change.
In 99% of the case, it is advised to have the timeout for the fuzzer. This is because we do not want the fuzzer to stop forever just because the target has hit a path that resulted in a infinite-loop.
## What changed
You do not have to wrap the executor with `TimeoutExecutor` anymore. You can just use `InProcessExecutor::new()` to instantiate the executor with the default timeout or use `InProcessExecutor::timeout(duration)` to start the executor with the customized duration of timeout.
The sugar crate abstracts away most of the complexity of LibAFL's API.
Instead of high flexibility, it aims to be high-level and easy-to-use.
It is not as flexible as stitching your fuzzer together from each individual component, but allows you to build a fuzzer with minimal lines of code.
To see it in action, take a look at the [`libfuzzer_stb_image_sugar` example fuzzer](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/libfuzzer_stb_image_sugar).
To see it in action, take a look at the [`libfuzzer_stb_image_sugar` example fuzzer](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers/inprocess/libfuzzer_stb_image_sugar).
This a proc-macro crate paired with the `libafl` crate.
At the moment, it just exposes the `derive(SerdeAny)` macro that can be used to define Metadata structs, see the section about [Metadata](../design/metadata.md) for details.
[Nyx](https://nyx-fuzz.com/) is a KVM-based snapshot fuzzer. `libafl_nyx` adds these capabilities to LibAFL. There is a specific section explaining usage of libafl_nyx [later in the book](../advanced_features/nyx.md).
@ -11,27 +11,27 @@ The first step is to download LibAFL and all dependencies that are not automatic
> previous command. Additionally, PowerShell-specific examples will use `>`
> rather than `$`.
While you technically do not need to install LibAFL, but can use the version from crates.io directly, we do recommend to download or clone the GitHub version.
While technically you do not need to install LibAFL, but can use the version from crates.io directly, we do recommend to download or clone the GitHub version.
This gets you the example fuzzers, additional utilities, and latest patches.
One of the external dependencies of LibAFL is the Clang C/C++ compiler.
While most of the code is in pure Rust, we still need a C compiler because stable Rust still does not support features that some parts of LibAFL may need, such as weak linking, and LLVM builtins linking.
While most of the code is written in pure Rust, we still need a C compiler because stable Rust still does not support features that some parts of LibAFL may need, such as weak linking, and LLVM builtins linking.
For these parts, we use C to expose the missing functionalities to our Rust codebase.
In addition, if you want to perform source-level fuzz testing of C/C++ applications,
If you do not have Rust installed, you can easily follow the steps described [here](https://www.rust-lang.org/tools/install)
to install it on any supported system.
Be aware that Rust versions shipped with Linux distributions may be outdated, LibAFL always targets the latest `stable` version available via `rustup upgrade`.
Be aware that Rust versions shipped with Linux distributions may be outdated, LibAFL always targets the latest `stable` version available via `rustup update`.
@ -4,10 +4,10 @@ Fuzzers are important tools for security researchers and developers alike.
A wide range of state-of-the-art tools like [AFL++](https://github.com/AFLplusplus/AFLplusplus), [libFuzzer](https://llvm.org/docs/LibFuzzer.html) or [honggfuzz](https://github.com/google/honggfuzz) are available to users. They do their job in a very effective way, finding thousands of bugs.
From the perspective of a power user, however, these tools are limited.
Their design does not treat extensibility as a first-class citizen.
Their designs do not treat extensibility as a first-class citizen.
Usually, a fuzzer developer can choose to either fork one of these existing tools, or to create a new fuzzer from scratch.
In any case, researchers end up with tons of fuzzers, all of which are incompatible with each other.
Their outstanding features cannot just be combined for new projects.
Their outstanding features cannot just be combined for new projects.
By reinventing the wheel over and over, we may completely miss out on features that are complex to reimplement.
To tackle this issue, we created LibAFL, a library that is _not just another fuzzer_, but a collection of reusable pieces for individual fuzzers.
@ -24,11 +24,11 @@ Some highlight features currently include:
This means it does not require a specific OS-dependent runtime to function.
Define an allocator and a way to map pages, and you are good to inject LibAFL in obscure targets like embedded devices, hypervisors, or maybe even WebAssembly?
- `adaptable`: Given years of experience fine-tuning *AFLplusplus* and our academic fuzzing background, we could incorporate recent fuzzing trends into LibAFL's design and make it future-proof.
To give an example, as opposed to old-skool fuzzers, a `BytesInput` is just one of the potential forms of inputs:
To give an example, as opposed to old-school fuzzers, a `BytesInput` is just one of the potential forms of inputs:
feel free to use and mutate an Abstract Syntax Tree instead, for structured fuzzing.
- `scalable`: As part of LibAFL, we developed `Low Level Message Passing`, `LLMP` for short, which allows LibAFL to scale almost linearly over cores. That is, if you chose to use this feature - it is your fuzzer, after all.
Scaling to multiple machines over TCP is also possible, using LLMP's `broker2broker` feature.
- `fast`: We do everything we can at compile time so that the runtime overhead is as minimal as it can get.
- `bring your own target`: We support binary-only modes, like QEMU-Mode and Frida-Mode with ASAN and CmpLog, as well as multiple compilation passes for sourced-based instrumentation.
- `bring your own target`: We support binary-only modes, like (full-system) QEMU-Mode and Frida-Mode with ASan and CmpLog, as well as multiple compilation passes for sourced-based instrumentation.
Of course, we also support custom instrumentation, as you can see in the Python example based on Google's Atheris.
- `usable`: This one is on you to decide. Dig right in!
Configurations for individual fuzzer nodes are relevant for multi node fuzzing.
The chapter describes how to run nodes with different configurations
in one fuzzing cluster.
This allows, for example, a node compiled with ASAN, to know that it needs to rerun new testcases for a node without ASAN, while the same binary/configuration does not.
This allows, for example, a node compiled with ASan, to know that it needs to rerun new testcases for a node without ASan, while the same binary/configuration does not.
Fuzzers with the same configuration can exchange Observers for new testcases and reuse them without rerunning the input.
A different configuration indicates, that only the raw input can be exchanged, it must be rerun on the other node to capture relevant observations.
LibAFL offers a standard mechanism for message passing over processes and machines with a low overhead.
LibAFL offers a standard mechanism for message passing between processes and machines with a low overhead.
We use message passing to inform the other connected clients/fuzzers/nodes about new testcases, metadata, and statistics about the current run.
Depending on individual needs, LibAFL can also write testcase contents to disk, while still using events to notify other fuzzers, using an `OnDiskCorpus`.
Depending on individual needs, LibAFL can also write testcase contents to disk, while still using events to notify other fuzzers, using the `CachedOnDiskCorpus` or similar.
In our tests, message passing scales very well to share new testcases and metadata between multiple running fuzzer instances for multi-core fuzzing.
Specifically, it scales _a lot_ better than using memory locks on a shared corpus, and _a lot_ better than sharing the testcases via the filesystem, as AFL traditionally does.
@ -12,7 +12,7 @@ The `EventManager` interface is used to send Events over the wire using `Low Lev
## Low Level Message Passing (LLMP)
LibAFL comes with a reasonably lock-free message passing mechanism that scales well across cores and, using its *broker2broker* mechanism, even to connected machines via TCP.
LibAFL comes with a reasonably lock-free message passing mechanism that scales well across cores and, using its _broker2broker_ mechanism, even to connected machines via TCP.
Most example fuzzers use this mechanism, and it is the best `EventManager` if you want to fuzz on more than a single core.
In the following, we will describe the inner workings of `LLMP`.
@ -28,12 +28,12 @@ Shared maps, called shared memory for the sake of not colliding with Rust's `map
Each client, usually a fuzzer trying to share stats and new testcases, maps an outgoing `ShMem` map.
With very few exceptions, only this client writes to this map, therefore, we do not run in race conditions and can live without locks.
The broker reads from all client's `ShMem` maps.
It checks all incoming client maps periodically and then forwards new messages to its outgoing broadcast-`ShMem`, mapped by all connected clients.
It periodically checks all incoming client maps and then forwards new messages to its outgoing broadcast-`ShMem`, mapped by all connected clients.
To send new messages, a client places a new message at the end of their shared memory and then updates a static field to notify the broker.
Once the outgoing map is full, the sender allocates a new `ShMem` using the respective `ShMemProvider`.
It then sends the information needed to map the newly-allocated page in connected processes to the old page, using an end of page (`EOP`) message.
Once the receiver maps the new page, flags it as safe for unmapping from the sending process (to avoid race conditions if we have more than a single EOP in a short time), and then continues to read from the new `ShMem`.
Once the receiver maps the new page, it flags it as safe for unmapping by the sending process (to avoid race conditions if we have more than a single EOP in a short time), and then continues to read from the new `ShMem`.
The schema for client's maps to the broker is as follows:
@ -54,10 +54,10 @@ After the broker received a new message from clientN, (`clientN_out->current_id
The clients periodically, for example after finishing `n` mutations, check for new incoming messages by checking if (`current_broadcast_map->current_id != last_message->message_id`).
While the broker uses the same EOP mechanism to map new `ShMem`s for its outgoing map, it never unmaps old pages.
This additional memory overhead serves a good purpose: by keeping all broadcast pages around, we make sure that new clients can join in on a fuzzing campaign at a later point in time
This additional memory resources serve a good purpose: by keeping all broadcast pages around, we make sure that new clients can join in on a fuzzing campaign at a later point in time.
They just need to re-read all broadcasted messages from start to finish.
So the outgoing messages flow like this over the outgoing broadcast `Shmem`:
So the outgoing messages flow is like this over the outgoing broadcast `Shmem`:
```text
[broker]
@ -72,13 +72,13 @@ So the outgoing messages flow like this over the outgoing broadcast `Shmem`:
[client0] [client1] ... [clientN]
```
To use `LLMP` in LibAFL, you usually want to use an `LlmpEventManager` or its restarting variant.
To use `LLMP` in LibAFL, you usually want to use an `LlmpRestartingEventManager` or its restarting variant.
They are the default if using LibAFL's `Launcher`.
If you should want to use `LLMP` in its raw form, without any `LibAFL` abstractions, take a look at the `llmp_test` example in [./libafl/examples](https://github.com/AFLplusplus/LibAFL/blob/main/libafl/examples/llmp_test/main.rs).
If you should want to use `LLMP` in its raw form, without any `LibAFL` abstractions, take a look at the `llmp_test` example in [./libafl/examples](https://github.com/AFLplusplus/LibAFL/blob/main/libafl_bolts/examples/llmp_test/main.rs).
You can run the example using `cargo run --example llmp_test` with the appropriate modes, as indicated by its help output.
First, you will have to create a broker using `LlmpBroker::new()`.
Then, create some `LlmpClient``s` in other threads and register them with the main thread using `LlmpBroker::register_client`.
Then, create some `LlmpClient`s in other threads and register them with the main thread using `LlmpBroker::register_client`.
@ -4,18 +4,18 @@ Multiple fuzzer instances can be spawned using different ways.
## Manually, via a TCP port
The straightforward way to do Multi-Threading is to use the `LlmpRestartingEventManager`, specifically to use `setup_restarting_mgr_std`.
It abstracts away all the pesky details about restarts on crash handling (for in-memory fuzzers) and multi-threading.
The straightforward way to do Multi-Threading is to use the [`LlmpRestartingEventManager`](https://docs.rs/libafl/latest/libafl/events/llmp/restarting/struct.LlmpRestartingEventManager.html), specifically to use [`setup_restarting_mgr_std`](https://docs.rs/libafl/latest/libafl/events/llmp/restarting/fn.setup_restarting_mgr_std.html).
It abstracts away all the pesky details about restarts on crash handling (for in-process fuzzers) and multi-threading.
With it, every instance you launch manually tries to connect to a TCP port on the local machine.
If the port is not yet bound, this instance becomes the broker, itself binding to the port to await new clients.
If the port is not yet bound, this instance becomes the broker, binding itself to the port to await new clients.
If the port is already bound, the EventManager will try to connect to it.
The instance becomes a client and can now communicate with all other nodes.
Launching nodes manually has the benefit that you can have multiple nodes with different configurations, such as clients fuzzing with and without ASAN.
Launching nodes manually has the benefit that you can have multiple nodes with different configurations, such as clients fuzzing with and without `ASan`.
While it's called "restarting" manager, it uses `fork` on Unix operating systems as optimization and only actually restarts from scratch on Windows.
While it's called "restarting" manager, it uses `fork` on Unix-like operating systems as optimization and only actually restarts from scratch on Windows.
## Automated, with Launcher
@ -23,7 +23,7 @@ While it's called "restarting" manager, it uses `fork` on Unix operating systems
The Launcher is the lazy way to do multiprocessing.
You can use the Launcher builder to create a fuzzer that spawns multiple nodes with one click, all using restarting event managers and the same configuration.
To use launcher, first you need to write an anonymous function `let mut run_client = |state: Option<_>, mut mgr, _core_id|{}`, which uses three parameters to create individual fuzzer. Then you can specify the `shmem_provider`,`broker_port`,`monitor`,`cores` and other stuff through `Launcher::builder()`:
To use launcher, first you need to write an anonymous function `let mut run_client = |state: Option<_>, mut mgr, _core_id|{}`, which uses three parameters to create an individual fuzzer. Then you can specify the `shmem_provider`,`broker_port`,`monitor`,`cores` and other stuff through `Launcher::builder()`:
```rust,ignore
Launcher::builder()
@ -42,18 +42,22 @@ To use launcher, first you need to write an anonymous function `let mut run_clie
This first starts a broker, then spawns `n` clients, according to the value passed to `cores`.
The value is a string indicating the cores to bind to, for example, `0,2,5` or `0-3`.
For each client, `run_client` will be called.
On Windows, the Launcher will restart each client, while on Unix, it will use `fork`.
If the launcher uses `fork`, it will hide child output, unless the settings indicate otherwise, or the `LIBAFL_DEBUG_OUTPUT` env variable is set.
On Windows, the Launcher will restart each client, while on Unix-alikes, it will use `fork`.
Advanced use-cases:
1. To connect multiple nodes together via TCP, you can use the `remote_broker_addr`. this requires the `llmp_bind_public` compile-time feature for `LibAFL`.
2. To use multiple launchers for individual configurations, you can set `spawn_broker` to `false` on all but one.
2. To use multiple launchers for individual configurations, you can set `spawn_broker` to `false` on all instances but one.
3. Launcher will not select the cores automatically, so you need to specify the `cores` that you want.
4. On `Unix`, you can chose between a forking and non-forking version of Launcher by setting the `fork` feature in LibAFL. Some targets may not like forking, but it is faster than restarting processes from scratch. Windows will never fork.
5. For simple debugging, first set the `LIBAFL_DEBUG_OUTPUT` env variable to see if a child process printed anything.
6. For further debugging of fuzzer failures, it may make sense to replace `Launcher` temporarily with a [`SimpleEventManager`](https://docs.rs/libafl/latest/libafl/events/simple/struct.SimpleEventManager.html#method.new) and call your harness fn (`run_client(None, mgr, 0);`) directly, so that fuzzing runs in the same thread and is easier to debug, before moving back to `Launcher` after the bugfix.
For more examples, you can check out `qemu_launcher` and `libfuzzer_libpng_launcher` in [`./fuzzers/`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers).
## Other ways
The `LlmpEventManager` family is the easiest way to spawn instances, but for obscure targets, you may need to come up with other solutions.
The `LlmpRestartEventManager` is the easiest way to spawn instances, but for obscure targets, you may need to come up with other solutions.
LLMP is even, in theory, `no_std` compatible, and even completely different EventManagers can be used for message passing.
If you are in this situation, please either read through the current implementations and/or reach out to us.
make -C ../../../../FreeRTOS/FreeRTOS/Demo/CORTEX_M3_MPS2_QEMU_GCC clean && make -C ../../../../FreeRTOS/FreeRTOS/Demo/CORTEX_M3_MPS2_QEMU_GCC $1=1IGNORE_INTERRUPTS=$IGNORE_INTERRUPTSIGNORE_BYTES=$IGNORE_BYTESIGNORE_INTERNAL_STATE=$IGNORE_INTERNAL_STATEINSERT_WC=$INSERT_WC$EXTRA_MAKE_ARGS
connection.execute("INSERT INTO combos (casename, toolname, fullname) VALUES (?, ?, ?)",(case,tool,format!("{}${}",case,tool))).unwrap();
connection.execute(&format!("DROP TABLE IF EXISTS {}${}",case,tool),()).unwrap();
connection.execute(&format!("CREATE TABLE IF NOT EXISTS {}${} (timestamp INTEGER PRIMARY KEY, min INTEGER, max INTEGER, median REAL, mean REAL, sdiv REAL)",case,tool),()).unwrap();
// Start a transaction
lettransaction=connection.transaction().unwrap();
letmutstmt=transaction.prepare(&format!(
"INSERT INTO {}${} (timestamp , min , max , median , mean , sdiv ) VALUES (?, ?, ?, ?, ?, ?)",
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.