With the rapid development of modern web APIs, developers must balance quality, reliability and security with time to market. When approaching testing, developers will aim for covering the “happy path”, that is, testing that the API behaves correctly under a strict set of conditions. While happy path testing provides some degree of confidence, one can expect any number of issues once the API is exposed to consumers.

API developers can try to find these issues early with negative testing, but without sufficient planning, negative testing is often ignored. This means that there are any number of unknown issues of unknown severity hidden in every API that may lead to benign errors, or at worse, serious security holes.

There are many tools that can be used to to fill in this gap, by “shifting left” the testing of APIs, using inputs that are meant to elicit insecure or unexpected behavior. One of these tools is ForAllSecure’s Mayhem for API.

Mayhem for API automates testing REST APIs by bringing the full might of fuzzing methodology to API testing. With the guidance of an API specification, Mayhem for API provides accurate and informative test coverage tailored to any REST API.

Another tool in this space is OWASP ZAP. ZAP is an open-source web application security scanner that can be used by both those new to application security as well as professional penetration testers. ZAP API scan is a script packaged with ZAP Docker images tuned for performing active scans against APIs. It is tuned to APIs, so it doesn’t bother looking for things like XSS

Both tools share the same goal:  Autonomously scan APIs to find security and implementation issues

In order to know what tool is right for you, we have put together some examples to see how both work with an example vulnerable REST API. Let’s begin! 

Prerequisites

• Docker (with docker compose configured)
• A Mayhem for API account. Sign up for Free! 
• Mayhem for API cli, mapi, installed

The Target API

We have chosen VAmPI as our Target API.

VAmPI is a vulnerable API made with Flask and it includes vulnerabilities from the OWASP top 10 vulnerabilities for APIs.

It was created by GitHub user erev0s as a vulnerable API to evaluate the efficiency of tools used to detect security issues in APIs

Clone the VAmPI source from GitHub: https://github.com/erev0s/VAmPI

git clone https://github.com/erev0s/VAmPI.git 

docker build -t vampi_docker:latest -f VAmPI/Dockerfile VAmPI 

docker compose -f VAmPI/docker-compose.yaml up -d

curl --location --request POST "http://localhost:5002/users/v1/register" \
--header 'Content-Type: application/json' \
--data-raw '{
"email":"foo@example.com",
"username": "foo",
"password": "bar"
}'

curl --location --request POST "http://localhost:5002/users/v1/login" \
--header 'Content-Type: application/json' \
--data-raw '{
"username": "foo",
"password": "bar"
}'

{
"auth_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NjIwNjkxNTIsImlhdCI6MTY2MjA2OTA5Miwic3ViIjoiZm9vIn0.7aB_94z7FmcGYNTaL67DW47Ht2WTBKlQ85eCbvmlBLM",
"message": "Successfully logged in.",
"status": "success"
}

export VAMPI_TOKEN="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NjIwNjkxNTIsImlhdCI6MTY2MjA2OTA5Miwic3ViIjoiZm9vIn0.7aB_94z7FmcGYNTaL67DW47Ht2WTBKlQ85eCbvmlBLM"

mapi run vampi auto ./VAmPI/openapi_specs/openapi3.yml \
--ignore-endpoint "^GET /createdb$" \
--header-auth "Authorization: token ${VAMPI_TOKEN}" \
--url "http://localhost:5002" \
--html mapi-report.html

Note some of the options used:

•  --header-auth "Authorization: token eyJ0eX…

• This uses the API token from the login. When scanning an API, you will be able to uncover more issues if you can successfully authenticate with it.

• --ignore-endpoint "^GET /createdb$" \

• The /createdb  endpoint should really only be called at most once per run to pre-populate the database. If it is called by the scanner, the results will be flaky as it recreates the database on every call.

• --html mapi-report.html

• This will create an html report with the results of your run.

At the end of the scan, your results will appear:

-Issues-
RULE_ID SUMMARY METHOD PATH
sql-injection GET /users/v1/{username}
internal-server-error AttributeError GET /books/v1
internal-server-error sqlite3.IntegrityError POST /books/v1
internal-server-error AttributeError GET /books/v1/{book_title}
internal-server-error sqlite3.OperationalError GET /users/v1/{username} 

docker compose -f VAmPI/docker-compose.yaml down 

docker compose -f VAmPI/docker-compose.yaml up -d

curl --location --request POST "http://localhost:5002/users/v1/register" \
--header 'Content-Type: application/json' \
--data-raw '{
"email":"foo@example.com",
"username": "foo",
"password": "bar"
}'

curl --location --request POST "http://localhost:5002/users/v1/login" \
--header 'Content-Type: application/json' \
--data-raw '{
"username": "foo",
"password": "bar"
}'

export VAMPI_TOKEN= &ltresult from previous request &gt

mkdir zap_working_dir

cp ./VAmPI/openapi_specs/openapi3.yml zap_working_dir

sed -i '' 's/localhost:5000/localhost:5002/g' zap_working_dir/openapi3.yml 

docker run --network=host -v $(pwd)/zap_working_dir:/zap/wrk \
-t owasp/zap2docker-weekly zap-api-scan.py \
-t "openapi3.yml" \
-f openapi -d \
-r zap-vampi.html \
-z "-config replacer.full_list\\(0\\).description=auth1
-config replacer.full_list\\(0\\).enabled=true
-config replacer.full_list\\(0\\).matchtype=REQ_HEADER
-config replacer.full_list\\(0\\).matchstr=Authorization
-config replacer.full_list\\(0\\).regex=false
-config replacer.full_list\\(0\\).replacement=token\\ ${VAMPI_TOKEN}
-config globalexcludeurl.url_list.url.regex='createdb$'"

Once the scan completes, you can find the results under ./zap_working_dir/zap-vampi.html.

‍

Comparing the Tools

Results

We have collected the results from both tools and will dig into what was found by each. These results were obtained using the latest version of the software products as of September 1, 2022.

‍

‍

‍

Execution

Results aren’t everything… both ZAP - API Scan and Mayhem for API are designed to be executed from the command line. Mayhem for API has been designed with an approachable command line user experience. Let’s compare how certain options, used in these examples, are expressed by each tool in the command line.  

‍

‍

Conclusion

Both Mayhem for API and ZAP (with ZAP - API Scan) can be used to autonomously scan your REST APIs for quality and security issues. 

ZAP will help you to discover some inconsistencies in your API such as leaking server version information via HTTP Response Header, or returning Unexpected Content-Type. 

Mayhem for API will more consistently find high severity issues, such as SQL Injection - and helps you keep your specification honest with your implementation by identifying issues where the API is not behaving how it is defined in the spec. 

With command line options for setting header bearer token, endpoint exclusion/inclusion and no need to modify the OpenAPI spec URL prior to running - we believe that Mayhem for API is easier to get up and running against your APIs.

While we think that Mayhem for API is all you need for API scanning needs, we also understand the value provided by ZAP for organizations that are looking to cover all their bases. 

With Mayhem for API, you can include results from ZAP - API Scan in every run with the –zap command-line argument! 

mapi run … --zap

 mapi run … --zap_import_json_results <path-to-zap-results.json>

Try it yourself!

Don’t just take our word for it, try it yourself!

Test this VAmPI and your own APIs by heading over to Mayhem for API to get started for free and start fuzzing!