Oops! Something went wrong while submitting the form.
Whenever Mayhem finds a crash, click on the test case ID to see more detail about the crashing test cases. The Mayhem UI should reveal a screen like so:
Here we can gain further insight into the behavior of the testme binary as a result of the particular input test case via the following tabs:
Defects: Shows a full list of defects exhibited by the particular test case.
Triage: Reveals metadata generated from triage for the particular test case (console output, backtrace, disassembly, register slate, signal number, memory maps).
Advanced Triage: Lists runtime errors as a result of performing advanced triage for the particular test case.
Mayhem also provides the necessary commands to re-execute the testme binary with the given input test case to reproduce the logged behavior.
Therefore, if we were to download and extract the testme binary as well as the crashing test case, and then execute the commands to reproduce the defect, we should see something similar to the following:
Info. Here we provide the testme binary for download, which requires a Linux OS to execute. If you have Docker installed, you can fuzz the containerized testme target within the Docker container itself. We will show you how to do this in the next lesson on CLI Fuzzing.
And there it is! We can see that the defect has been reproduced for the given input test case. Now, let's fix the underlying code, re-compile the testme binary, and execute another Mayhem run to confirm that the defect (and its associated test cases) are fixed!
Running Regression Testing and Confirming Fixes
When Mayhem generates test cases involved with fuzzing a target application, it also saves the test cases for future Mayhem runs of the same target. This way, future Mayhem runs can utilize those previously generated test cases to confirm if the current fuzzing behavior of the target application has changed (i.e. previous passing test cases now crash or previous crashing test cases now pass). This is called regression testing.
Mayhem will re-use the same test suite for future Mayhem runs of a given <project>/<target> run of a particular owner. For this example, Mayhem will re-use the generated test suite for the run forallsecure-tutorial/testme owned by your Mayhem user account.
Let's see how this works in practice. Recall that we just fuzzed the testme application that was shown to have an improper input validation defect:
If we scroll down to the bottom of the Mayhem run page, we can see the results of the regression test on a per test case basis. Notice how previously crashing test cases and their associated defects have been marked as fixed.
And that's it! Well done. You've now not only used Mayhem to find defects for a target application, but also confirmed that the test cases relating to the defects have been resolved upon fuzzing a fixed version of the target binary and it's underlying code!