Reproducing Generated Test Case Crashes in Mayhem

Mayhem Team
October 4, 2022
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Whenever Mayhem finds a crash, click on the test case ID to see more detail about the crashing test cases. The Mayhem UI should reveal a screen like so:

Here we can gain further insight into the behavior of the testme binary as a result of the particular input test case via the following tabs:

  1. Defects: Shows a full list of defects exhibited by the particular test case.
  2. Triage: Reveals metadata generated from triage for the particular test case (console output, backtrace, disassembly, register slate, signal number, memory maps).
  3. Advanced Triage: Lists runtime errors as a result of performing advanced triage for the particular test case.

Mayhem also provides the necessary commands to re-execute the testme binary with the given input test case to reproduce the logged behavior.

 ./root/tutorial/testme/v1/testme <TESTCASE_ID>


Therefore, if we were to download and extract the testme binary as well as the crashing test case, and then execute the commands to reproduce the defect, we should see something similar to the following:

 ~/tutorial/testme/v1# ./testme b86204ef1dc8cfc10654e1fe7b1d1c241ef7a0b1da67430b9865c328499d13e7 


Info. Here we provide the testme binary for download, which requires a Linux OS to execute. If you have Docker installed, you can fuzz the containerized testme target within the Docker container itself. We will show you how to do this in the next lesson on CLI Fuzzing.

And there it is! We can see that the defect has been reproduced for the given input test case. Now, let's fix the underlying code, re-compile the testme binary, and execute another Mayhem run to confirm that the defect (and its associated test cases) are fixed!

Running Regression Testing and Confirming Fixes

When Mayhem generates test cases involved with fuzzing a target application, it also saves the test cases for future Mayhem runs of the same target. This way, future Mayhem runs can utilize those previously generated test cases to confirm if the current fuzzing behavior of the target application has changed (i.e. previous passing test cases now crash or previous crashing test cases now pass). This is called regression testing.


Mayhem will re-use the same test suite for future Mayhem runs of a given <project>/<target> run of a particular owner. For this example, Mayhem will re-use the generated test suite for the run forallsecure-tutorial/testme owned by your Mayhem user account.

Let's see how this works in practice. Recall that we just fuzzed the testme application that was shown to have an improper input validation defect:

  1. CWE-20: Improper input validation defect.

Now, we'll want to fix the underlying defect and execute a regression test to fuzz the target with the previously generated test cases and confirm that the found defect has been fixed.

Let's take a look at a fixed version of our testme application.

#include <stdio.h>
#include <string.h>

int fuzzme(char *buf)
if(buf[0] == 'b')
if(buf[1] == 'u')
if(buf[2] == 'g') {
return 0; // Fixed: No more defect.
return 0;

int main(int argc, char *argv[])
FILE *f;
char buf[12];

if(argc != 2){
fprintf(stderr, "Must supply a text file\n");
return -1;
f = fopen(argv[1], "r");
if(f == NULL){
fprintf(stderr, "Could not open %s\n", argv[1]);
return -1;
if(fgets(buf, sizeof(buf), f) == NULL){
fprintf(stderr, "Could not read from %s\n", argv[1]);
return -1;
return 0;

Simply changing the abort(); to return 0; on line 9 should be enough to fix the improper input validation defect.

Within the create new run flow, click the Show Mayhemfile link at the bottom of the page to confirm that your Mayhemfile looks similar to the following:

duration: 90
project: forallsecure-tutorial
target: testme
- name: regression_testing
- cmd: /root/tutorial/testme/v2/testme @@

Once ready, go ahead and kick off the regression test for the recent fix of the testme application!

Development Speed or Code Security. Why Not Both?

Mayhem is an award-winning AI that autonomously finds new exploitable bugs and improves your test suites.

Get Mayhem Free Request A Demo

If we scroll down to the bottom of the Mayhem run page, we can see the results of the regression test on a per test case basis. Notice how previously crashing test cases and their associated defects have been marked as fixed.

And that's it! Well done. You've now not only used Mayhem to find defects for a target application, but also confirmed that the test cases relating to the defects have been resolved upon fuzzing a fixed version of the target binary and it's underlying code!

Share this post

Fancy some inbox Mayhem?

Subscribe to our monthly newsletter for expert insights and news on DevSecOps topics, plus Mayhem tips and tutorials.

By subscribing, you're agreeing to our website terms and privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Add Mayhem to Your DevSecOps for Free.

Get a full-featured 30 day free trial.

Complete API Security in 5 Minutes

Get started with Mayhem today for fast, comprehensive, API security. 

Get Mayhem

Maximize Code Coverage in Minutes

Mayhem is an award-winning AI that autonomously finds new exploitable bugs and improves your test suites.

Get Mayhem