ForAllSecure's API Fuzzer: Mayhem for API

Develop reliable APIs, faster.

Api Fuzzer

What is Mayhem for API?

Mayhem for API is an API Fuzzer that provides accurate and automated test coverage for APIs in your application portfolio. The API fuzzer was designed to be easy to use and configure, bringing the full might of fuzzing methodology to API testing without the roadblocks typically encountered.

How does it work?

Mayhem for API uses a fuzzing engine to automatically generate a comprehensive suite of inputs used to test function and robustness of an application’s API infrastructure. By using fuzzing techniques to generate inputs and observing the response from the application, the API fuzzer can quickly iterate through multitudes of test cases to find weakness in an API’s functionality or security.

As you'll see, setting up the fuzzer is a breeze: all you need to do is upload the API specification of the application to test, and a link to the target application that the fuzzer can reach to start testing. Currently, the API fuzzer provides a status of all the endpoints tested in the application, and a summary of the response codes observed with each input.

Next Steps

We have a lot planned in the future to improve and evolve Mayhem for API. We would like to thank you for joining us in the beginning of this exciting product and we look forward to learning from you and creating an indispensable part of your application development and testing practice.

Reach out!

We aim to provide a great experience. Please reach out to us on Discord or by emailing mayhem4api@forallsecure.com if you have any suggestions or run into any issues. We're more than happy to help!

Getting Started

Let’s start your API fuzzing journey! There’s a lot to learn, but every journey starts somewhere. In this chapter, we’ll discuss:

  • Installing the Mayhem for API command-line client on Linux, macOS, or Windows.
  • Using the CLI to fuzz our first API
  • Monitoring progress

Signing up

If you haven't done so yet, the fastest way to get started is to sign up for a free plan at https://mayhem4api.forallsecure.com/signup. If you already have an account, then you are ready to go for the next steps!

Installation

The Mayhem for API CLI is available to download for various common platforms.

ℹ️ The CLI will automatically keep itself updated when used as we make fixes and bug improvements.

MacOS

curl -Lo mapi https://mayhem4api.forallsecure.com/downloads/cli/latest/macos/mapi \
  && chmod +x mapi

Here's an easy way to add the mapi executable to your path:

sudo mv mapi /usr/local/bin

Linux (64-bit)

curl -Lo mapi https://mayhem4api.forallsecure.com/downloads/cli/latest/linux-musl/mapi \
  && chmod +x mapi

Here's an easy way to add the mapi executable to your path:

sudo mkdir -p /usr/local/bin/
sudo install mapi /usr/local/bin/

Windows (64-bit)

From a Windows 10 terminal (PowerShell or cmd):

curl.exe -Lo mapi.exe https://mayhem4api.forallsecure.com/downloads/cli/latest/windows-amd64/mapi.exe

or download :

https://mayhem4api.forallsecure.com/downloads/cli/latest/windows-amd64/mapi.exe

Test it out!

Make sure the CLI works by running:

mapi --help

Authentication

The mapi CLI communicates with our API using OAuth 2.0 Bearer Tokens. The token will be read by the environment variable, MAPI_TOKEN, if available.

To get a new token, login with the CLI and follow the prompts:

$ mapi login

Enter username or email: mapi_fuzzer
Enter a password: [hidden]

Welcome to Mayhem for API! We have saved a new API token in
your local settings at '/Users/mapi_fuzzer/Library/Preferences/rs.mapi/mapi.toml':

           3BzW...

Setting the displayed API Token to the environment variable, MAPI_TOKEN, will allow you to run the CLI on other computers, such as part of your Continuous Integration build.

Now you can try contacting the API. Let's get the list of targets to which you have access:

mapi target list

You should see an (empty) list of API targets. Let's add our first target so that list won't stay empty for long.

Hello, Fuzz!

We have prepared an instance of the Swagger Petstore API for you to try fuzzing. The Swagger Petstore is a stand-alone REST API server which implements the OpenAPI 3 Specification.

Fuzz the Swagger Pestore API for 20 seconds by running the following in your terminal...

mapi run \
     petstore 20 'https://demo-api.mayhem4api.forallsecure.com/api/v3/openapi.json' \
     --url 'https://demo-api.mayhem4api.forallsecure.com/api/v3/' \
     --interactive

...once the run completes, visit your dashboard to see the results!

What just happened?

Let's break that command down, to see what happened:

mapi run

Use the CLI to run a fuzzing job.

ℹ️ Check out mapi help if you want to explore some of the other things the CLI can do!

petstore

Use (and create, if it doesn't already exist) a "target" named petstore for this run. We'll come back to targets in more detail later. For now, a short name that uniquely identifies the API being tested is perfect.

ℹ️ Targets are used to group related fuzzing jobs.

20

Run this fuzzing job for just 20 seconds.

https://demo-api.mayhem4api.forallsecure.com/api/v3/openapi.json

URL of the API specification.

The fuzzer currently accepts either OpenAPI 3.0 specification, Swagger 2.0 specification or Postman 2.x collections in json or yaml format. HTTP Archive (HAR) is also supported by converting a .HAR file into an OpenAPI 3.0 specification.

ℹ️ You may also use an API specification file from your local file system.

--url 'https://demo-api.mayhem4api.forallsecure.com/api/v3/'

Tells the fuzzer, which is running locally on your computer, to address its tests to the petstore demo API on our network. This method will work for APIs that are:

  • Behind a corporate firewall
  • On localhost
  • In a private network
  • On the public internet

If the machine running the CLI can access the API - then we can fuzz it!

Local Fuzz

--interactive

Runs the fuzzing job with an interactive, terminal-based user interface. This option should be omitted in a continuous integration environment.

You will see the following UI in interactive mode:

Status UI

The status UI contains a fair amount of information:

  • A progress bar at the top, based on the duration of the job and elapsed time.
  • The job logs. If the job doesn't start properly, or runs into trouble, this is where you should see information on how to resolve the issue.
  • The status codes that the fuzzer observed per endpoint. This will help you determine if the fuzzer is covering the endpoints and status code you expected. For instance, if you see only 401 responses, then that means the fuzzer probably needs help with authentication. We cover this in more details in “Troubleshooting".

The UI has a few navigation keys that might be handy:

  • q, ^C or <Esc> to quit
  • <Tab> to switch focus between widgets
  • <Space> to make the current widget full screen
  • Arrow keys for horizontal and vertical scrolling

In your run you should see a number of red endpoints which means that at least one 500 response code is observed. That means we just found some bugs! You should also see a number of green endpoints. These mean that at least one 2xx response has been observed.

ℹ️ We won't cover authentication just yet but we will cover it later in the API Authentication chapter.

Summary

You are already off to a great start on your API fuzzing journey! In this chapter you learned how to:

  • Install the latest version of the Mayhem for API CLI
  • Fuzz APIs and monitor fuzzing progress

This is a great time to talk about the main output of fuzzing: bugs. You just found a few of them in our demo API, so let's figure out how to dig into them a bit more.

Organizations

Organizations are used to share your API Targets and Run results with other users. You may belong to any number of Organizations. As a Organization owner, you can invite others to join you and create service accounts to integrate Mayhem for API into your DevOps pipeline.

When you sign up you will be prompted to create a new Organization. You will be the Owner of this Organization.

Organization Roles

Organization members may have one of the following roles:

Owner: Organization Owners have full control over Organizations and Members

Member: Organization Members may create API Targets and view Job results, but may not manage other members.

ActionOwnerMember
Create organization invitationX
Remove memberX ( ✔ can remove self )
Create Service AccountsX
Create Target
Update API Target
View Jobs
Change member role

You can change the roles of Organization members with the mapi organization set-role command.

Inviting others to join your Organization

Organization Owners can create a new invitation with the mapi organization invite command using the name of their Organization.

Invitations are only valid for one claim and will expire after two days by default. You can extend the number of claims and duration with the --max-claims and expiration-min options respectively. For example:

mapi organization invite --max-claims 5 --expiration-min 2880 my-organization

This will present you a message that you can send to whomever you would like to invite to your Organization via Email/Slack/SMS/etc:

A new invitation has been created for organization <your-organization>>!

Up to 1 user(s) can join the organization by going to 'https://mayhem4api.forallsecure.com/join/uGb79...'.

The invitation link is valid for 1 day.

Anyone that accepts your invite will join with the Member role.

Service Accounts

Organization owners may create service accounts for accessing the API as part of automation to avoid using API Tokens from individual accounts. You may create as many service accounts as you require in your organization.

Service accounts do not count toward developer count for the purpose of billing.

To create a new service account, use the mapi CLI:

mapi organization service-account create <my-organization> <service-account-name>

The service account will be added to your organization - and an API token will be presented once. You must record this token as it will not be shown again.

New Service Account named '<service-account-name>' created! A new API token has been
created.

           SWOJg...

You can use this token in the environment variable, MAPI_TOKEN, to access
the API Fuzzer API as the Service Account. We will not display this value
again.

You may now use the API token presented in the response to access targets and run fuzzing jobs in your Organization.

Removing a Service Account

To delete a service account (and invalidate its API Token) you must remove the account from your organization. Find the ID of your service account by listing organization members:

mapi organization list-members <my-organization>

 ID  Name               Role   
  1  foo                Owner  
  2  bar                Member  
  3  <service-account>  Member   <<<<<

Now remove the service account from your organization:

mapi organization remove-member <my-organization> <service-account-id>

Successfully removed Organization '<my-organization>>' member '3'!

HTTP Archive (HAR)

Not all APIs have an OpenAPI/Postman/Swagger specification available. The only specification is the code itself. It is still possible to fuzz an API without a specification by recording transactions with the API as an HTTP Archive (.har file) and converting the recording into an OpenAPI specification. While this enables fuzzing of APIs without specifications, there is a trade-off in the accuracy of the converted specification compared with one generated from the code or maintained by hand.

HAR Files may contain sensitive information

⚠️ HAR files contain everything that was sent between your client/browser and the target API. This may include Authorization headers, cookies, etc. It is not advisable to persist or convert a .har file unless it has been stripped of sensitive information beforehand.

Converting a .HAR to an OpenAPI 3.0 specification

ℹ️ Requires mapi 2.6.16 or newer

The mapi convert command is used to convert from other formats to an OpenAPI 3.0 specification.

For example, the following command converts a recording, recording.har into an OpenAPI spec, openapi.yaml:

mapi convert har recording.har --host '127.0.0.1:8080' --base-path 'api/v1' --out openapi.yaml

Let's examine the arguments:

  • --host '127.0.0.1:8080' The conversion will attempt to infer the URL of your API from the recording. If the recording contains requests to multiple URLs, you can specify the --host option to restrict conversion to requests that are only from the specified host.
  • --base-path 'api/v1' If your API has a common base path this option will strip the common prefix from all request URLs. This can significantly improve the accuracy of the converted spec.
  • --out opennapi.yaml This will overwrite any existing file, openapi.yaml, with the converted specification. Without this argument, the conversion will be printed to standard out in your terminal.

ℹ️ See all available options with the mapi convert --help command

Upon successful conversion you may inspect the converted file and make any modifications you feel are necessary or start a new run immediately.

Create a .HAR recording with your Browser

If you have a web frontend that interacts with your API, you can record an HTTP Archive by navigating your website and running through various use cases to build a recording of API interactions based on which features you access from the frontend.

Most modern browsers support recording your interactions and exporting a HAR file. See https://toolbox.googleapps.com/apps/har_analyzer/ for browser-specific details.

Create a .HAR recording with a proxy

A web frontend may not make sufficient requests to your API to generate a useful OpenAPI specification. You may have other tools such as a CLI, automated test suites, or a collection of manual curl invocations that can do this. In this case, you can use a proxy to record all of your traffic to produce a HAR file.

The following tools provide support for this capability:

💡 There are many other tools that can help you record your traffic to export as a HAR file. If you see any that should be included here, please let us know: mayhem4api@forallsecure.com

Identifying Buggy Endpoints

Mayhem for API reports any issues it finds as well as the input that triggers them. In this chapter, we describe the different solutions you can use to see the issues Mayhem for API finds.

Web UI

Mayhem for API uploads findings from each fuzzing run to the ForAllSecure cloud service. Sign in to the UI at https://mayhem4api.forallsecure.com, where you can dig into the details of each run. This includes the details of any issues found, as well as per-endpoint statistics such as response code distribution and latency percentiles. Here's what the web UI looks like:

Web UI

Detailed Report in the Web UI

HTML Reports

Mayhem for API can generate static, local HTML reports using the --html argument to mapi run. These reports contain essentially the same information as their web UI counterparts. They can be saved as "build artifacts" in a CI system. Here's what an HTML report looks like:

HTML Report

Detailed Static HTML Report

JUnit Reports

Mayhem for API can generate industry standard, CI-friendly JUnit reports with the --junit argument to mapi run. These reports contain all of the detailed issues found—but not the additional statistical details from web and HTML reports!—in a machine-readable form, which most CI software can consume and render natively.

SARIF Reports

Mayhem for API can also generate SARIF files with the --sarif argument to mapi run. SARIF files are a great way to upload issues to GitHub, if you happen to use GitHub. See our “GitHub Integration" section for more details.

GitHub

Mayhem for API Issues in GitHub

SARIF files can also be loaded in VSCode using a SARIF extension from Microsoft:

VSCode

Mayhem for API Issues in VSCode

ℹ️ To give accurate file and line information for each issue, Mayhem for API requires your API to include stacktraces in the body of "500 Internal Server Error" responses. Only do that in dev & testing -- never in production!

Debugging tools

Of course, having Mayhem for API find an issue is, in some ways, just the start. Isolating, reproducing and ultimately fixing issues is hard work!

Here are a few approaches we hope will make it easier. (These are all completely independent of Mayhem for API.)

API Logs

The simplest solution is to look through your API logs when a 500 is triggered. Logs often include stack traces pinpointing which line of code triggered the issue. If you have centralized logging infrastructure, with ElasticSearch or Splunk for instance, you'll likely be able to filter the logs by response code to only see 500 responses.

The fuzzer-generated requests will contain the following headers, which can help you look through your logs and correlate log events with specific fuzzing inputs:

  • User-agent: mapi/X.X.X, where X.X.X represents the mapi CLI version.
  • X-Mapi-Program-UUID: <UUID>, where UUID corresponds to a fuzing input. This header allows you to correlate buggy requests from the mapi reports to your API logs.

Error Monitoring Tools

We believe that installing an error monitoring tool in your API is the best way to dig into bugs. They can monitor not only your running APIs, but also any other backend systems like other microservices, background workers, and so on. Those backend systems might error out without the API returning a 500 responses, and error monitoring tools can help you detect those issues triggered but undetected by the fuzzer.

Error monitoring tools detect errors, gather as much context as possible, group similar errors together, and make them available to you through their UI, which looks something like this for sentry.io:

Sentry UI

Plenty of third-party companies offer great error monitoring tools:

Odds are your organization is already using such a tool in production. You can leverage it for fuzzing to get deep insights into the bugs that the fuzzer triggered. Your error monitoring tool will now alert you when the fuzzer finds an issue. It will give you plenty of context like the stack trace to understand the issues. You will be alerted of those issues perhaps even before the bug gets deployed in production!

Issue Rules (Checkers)

Issues discovered by Mayhem for API are categorized by Rule. Each Rule corresponds to one or more checkers that are executed as part of a fuzzing run.

The following is a list of supported Rules. Click a link below to view the details of a specific rule.

Suppressing Issues

Not all issues reported by Mayhem for API may be relevant or of interest to an API maintainer. Mayhem for API offers a mechanism to suppress issues discovered during fuzzing jobs.

The .mapi configuration file

A .mapi configuration file specifies rules for issue suppression:

#
# Example .mapi configuration file
#
version: "1.0"

# An array of rules for suppressing issues.
#
# If an issue matches any rule, it will be suppressed
# from reporting.
#
# Any issue will only match a rule if ALL statements for
# a rule match the issue.

suppressed:  
- reason: "Suppress Internal API issues"  
  path: /internal/.*  
- reason: "Ignore flask debug exceptions"
  summary: flask\.debughelpers\.*

An issue suppress rule is composed of regular expressions which match on different issue fields:

suppressed:
- reason: "(required string) Description of why issues matches this rule are suppressed"
  method: "(optional regex) GET|PUT|POST|DELETE|PATCH... etc. Matches HTTP Method"
  path: "(optional regex) /foo|/tools/.* Matches the HTTP request path"
  rule_id: "(optional regex) internal-server-error|auth-bypass|server-crash... Matches mapi rule id"
  summary: "(optional regex) matches additional issue summary - such as stacktrace message"
  

Where to put .mapi file?

💡 We recommended checking the .mapi file into source control so that you may track changes to suppressed rules with your SCM tool.

The mapi CLI will resolve the .mapi file with the following order of precedence:

  1. Set with the --config <path-to-file> when calling mapi run
  2. Set with the MAPI_CONFIG_FILE environment variable
  3. Located in the working directory where mapi run is called
  4. Located in the project root (adjacent to the .git folder, if one exists)

How do I see suppressed issues?

mapi will still upload suppressed issues during a run, but they will be hidden by default. In order to see suppressed issues from the CLI, for instance, you will need to supply an extra flag --include-suppressed:

▶ mapi issue list <job-id>> --include-suppressed
   
 ID     RULE_ID                SUMMARY  METHOD  PATH                 SUPPRESSED  SUPPRESSED_REASON
 35717  internal-server-error           GET     pet/findByTags       true        Ignore all GETs
 35716  internal-server-error           GET     pet/{petId}          true        Ignore all GETs
 35715  internal-server-error           POST    user                 false
 35714  internal-server-error           POST    pet                  false

Selective Route Testing

By default, Mayhem for API will fuzz every endpoint it finds in the provided spec.

If you need to filter down from this full set of endpoints, for example if:

  • you need to exclude a "logout" endpoint because it breaks all subsequent authenticated requests
  • you need to divide your spec into subsets with different authentication requirements
  • you want to incrementally add routes to mapi as the initial findings in each route are addressed

... or whatever other reason, the mapi command-line tool has you covered with a number of options.

Using OpenAPI Tags

If you already have an OpenAPI spec with tags at the right granularity to describe the subsets you want to filter on, you're in luck.

Use the --include-endpoints-by-tag <tag> option to specify that only endpoints with the given tag should be included (implicitly excluding all endpoints which don't have it.) If you specify multiple tags (by including the option more than once), they are logically "OR"ed together.

Use the --ignore-endpoints-by-tag <tag> option to specify that endpoints with the given tag should be explicitly excluded. If you specify multiple tags, they are logically "OR"ed together.

If you combine the two options, --include is applied first, followed by --ignore.

Using Endpoint Names

If you don't have tags at the right granularity, you can use powerful (but somewhat more complicated!) endpoint name matching to choose which endpoints to test.

ℹ️ This feature computes a unique "endpoint name" for each endpoint, which is just the HTTP verb and the path, joined with a space, like GET /ping or POST /users/{id}!

Use the --include-endpoint <pattern> option to specify that only endpoints whose computed name matches the given pattern should be included (implicitly excluding all endpoints which don't match it.) If you specify multiple patterns (by including the option more than once), they are logically "OR"ed together.

Use the --ignore-endpoint <pattern> option to specify that endpoints whose computed name matches the given pattern should be explicitly excluded. If you specify multiple patterns, they are logically "OR"ed together.

If you combine the two options, --include is applied first, followed by --ignore.

ℹ️ The "pattern" used to match endpoint names is a full-blown regular expression, if you need that kind of power.

API Target Authentication

Mayhem for API supports a number of ways to authenticate to the target API. Giving the fuzzer a way to authenticate to the target API will enable it to exercise more endpoints and maximize coverage.

Basic Authentication

Basic access authentication is a simple technique for enforcing access control to web resources, which is supported by most web servers. You can specify basic authentication credentials when you fuzz your target with mapi run with a command line option or environment variable.

CLI Option:

mapi run --basic-auth "username:password" <api-name> <duration> ./openapi.json

Environment:

export MAPI_BASIC_AUTH="username:password"
mapi run <api-name> <duration> ./openapi.json

Note that basic authentication does not protect the username and password by itself. They are simply encoded with base64 in transit, but not encrypted or hashed. Basic authentication should be used in conjunction with HTTPS to protect the credentials, or on a trusted network.

Bearer Tokens

Bearer authentication (also called token authentication) is an HTTP authentication scheme that involves security tokens called bearer tokens The client must send this token in the Authorization header when making requests to protected resources:

mapi run --header-auth 'Authorization: bearer <token>' <api-name> <duration> ./openapi.json

This method can also be used for any header-based authentication, for instance:

mapi run --header-auth 'X-Custom: auth <token>' <api-name> <duration> ./openapi.json

You can also use an environment variable to pass your token:

export MAPI_HEADER_AUTH="Authorization:Bearer <token>"
mapi run <api-name> <duration> ./openapi.json

Note that the authorization header does not protect the token by itself. It is not encrypted or hashed before sending it to the server. As with basic authentication, this authentication method should be used in conjunction with HTTPS to protect the credentials, or on a trusted network.

Note that if you would like to specify additional headers that do not include credentials, we recommend using -H instead of --header-auth. Mayhem for API treats --header-auth differently when looking for authentication bypasses.

Cookie Authentication

Cookie authentication uses HTTP cookies to authenticate client requests and maintain session information. Cookies are generally returned by the server after a successful login, and sent by the clients in subsequent requests. You can specify cookies when you fuzz your target with mapi run either via command line options or an environment variable.

CLI Option:

mapi run --cookie-auth "PHPSESSID=abe67cd" <api-name> <duration> ./openapi.json

Environment:

export MAPI_COOKIE_AUTH="PHPSESSID=abe67cd"
mapi run <api-name> <duration> ./openapi.json

Note that cookies are not encrypted or hashed before being sent to the server. As with basic and header authentication, this authentication method should be used in conjunction with HTTPS to protect the credentials, or on a trusted network.

More to come!

We are planning on adding more authentication methods soon, and we'd love to hear from you about what you'd like to see.

Logout blacklisting

If the credentials you passed to the fuzzer can be invalidated through a logout endpoint, consider blacklisting that endpoint with the --ignore-endpoint flag in mapi run, something like this:

mapi run [...] --ignore-endpoint "/api/logout"

For more details on this and other mechanisms for choosing which endpoints to fuzz, see Selective Route Testing.

CSRF Tokens

Note that the Mayhem for API does not currently handle CSRF tokens. Those are extremely common to prevent CSRF attacks when using cookie authentication, but often disabled when using API tokens. If this is something you'd like us support better, shoot us an email at mayhem4api@forallsecure.com

Configure your API

In this section we will examine how to configure specific frameworks to help Mayhem For API find problems, such as possible SQL Injection, that might not be as discoverable with default application settings. These configuration changes are optional and are rarely safe for production; however, they can provide more value out of Mayhem for API when enabled in a safe environment such as part of your CI/CD pipeline.

These changes typically involve adjusting your API configuration to return additional information in the response whenever an internal error occurs, such as the stack trace.

This will help you get to the root cause of buggy endpoints faster and can help the fuzzer identify security threats more accurately such as error-based SQL Injection attacks:

💡 If you do not see your API framework here yet, please let us know: mayhem4api@forallsecure.com

Spring Boot (Java)

Various properties can be specified inside your application.properties file, inside your application.yml file, or as command line switches. please visit https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html for a comprehensive list of properties.

For the following configurations, assume we are working with application.properties file.

⚠️ These settings are appropriate for development testing, but should not be used in production.

Include Stack Trace on Server Error

Update your application.properties file with the following line:

server.error.include-stacktrace=always

Whenever the fuzzer triggers an error, Spring Boot will include a stack trace in the response.

Flask (Python)

When developing a Flask-based API, the built-in Werkzeug development server provides a debugger which shows an interactive traceback in the browser when an unhandled error occurs during a request.

For more information, visit https://flask.palletsprojects.com/en/latest/debugging/.

⚠️ These settings are appropriate for development testing, but should not be used in production.

Include Traceback on Server Error

Either set the FLASK_ENV=development environment variable prior to running your app:

$ export FLASK_ENV=development
$ flask run

Or, include the debug=true flag when running from Python code

from flask import Flask
app = Flask(__name__)

if __name__ == '__main__':    
    app.run(debug=True)

Whenever the fuzzer triggers an error, Flask will include the traceback in the response.

CI Integration

We recommend running Mayhem for API as part of your CI pipelines. Adding the fuzzer should take only a few minutes. Running the fuzzer as part of your continuous build will automatically check your API with your latest changes before they are merged and deployed.

Downloading the CLI

The first step is to download the CLI. We recommend using the latest url to always get the latest release. We aim to retain backwards compatibility for command line arguments so that your build always works.

Assuming your build is running in Docker, we recommend adding the following lines to add the CLI to your Docker image:

ADD https://mayhem4api.forallsecure.com/downloads/cli/latest/linux-musl/mapi /usr/bin/mapi
RUN chmod 0755 /usr/bin/mapi

This ensures that you get the latest release every time, without affecting the Docker build cache for follow on layers if there hasn't been a new release. This will download the CLI (~25MB) every time you build the image. We recommend adding the line near the end of your build, shortly before you add your code, to minimize the number of uncached layers to rebuild when there is a new CLI release.

Optionally, to avoid re-downloading the CLI every time, we recommend using Docker's BuildKit. BuildKit checks the URL to see if a new version is available, and only downloads it if necessary. This ensures you get the latest release at all times without slowing down your builds with unnecessary downloads. Enable BuildKit with a simple environment variable:

$ DOCKER_BUILDKIT=1 docker build .

Finally, if you happen to use Docker's experimental syntax, you can download the CLI in one command:

ADD --chmod=755 https://mayhem4api.forallsecure.com/downloads/cli/latest/macos/mapi /usr/bin/mapi

Authentication

The CLI requires a token to function. We recommend storing your mapi token in your CI secret stores, and loading it in the MAPI_TOKEN environment variable. The CLI will use the value in that environment variable automatically.

Running the fuzzer

To fuzz your API, you'll have to first start your API locally following our best practices. Once it's up and running, invoke the fuzzer with:

mapi run <target> <duration-in-s> <spec-file> --url <url> --junit results.xml
  • <target> is the name of your api target that you created with mapi target create. The target name is often the name of your project.
  • <duration-in-s> depends on how long you want your build to take. We recommend a minimum of 60 seconds. To maximize fuzzing duration without slowing down the build, we recommend fuzzing in parallel to other build steps, for the duration of those build steps.
  • If your API requires authentication for some endpoints, consider passing it to the fuzzer for best results.

Note that --url and --junit options are optional. If --url is unspecified, it will come from the URL you specified in the API target. --junit is only necessary if you want your CI to parse a JUnit XML to extract the tests and potential failures (see below).

Gathering the results

The CLI will return an error code if any 500 responses are observed. This can be used to fail the build if you so choose. In addition, with the --junit option, the fuzzer will output a JUnit XML file that your CI can parse.

The JUnit will contain a sample request/response for any endpoint that returned a 500, which can be helpful to debug the issue (especially if the API is run in debug mode and the response contains debugging output like stacktraces).

To get a human-friendly report that you can save as a build artifact, use the --html option to get a report like the following:

Api Fuzzer

Ignoring Errors

You may filter out certain types of response errors that you do not want to appear in your results using the CLI. For example to prevent 500 Internal Server Error responses from being marked as issues, include the --ignore-rule InternalServerError flag in your call to mapi run.

The --ignore-rule flag may be used multiple times to ignore more than one type of error. By default, ALL detected response errors will be classified as issues.

mapi run <target> <duration-in-s> <url> --ignore-rule InternalServerError

💡 To see a full list of --ignore-rule options, see the output of mapi run --help.

CI-Specific examples

Not sure what's the best way to integrate the fuzzer with your CI? Reach out to use, and we'll add an example for your CI here.

GitHub Integration

Mayhem for API comes with a GitHub Action and a GitHub App to help you check every change to your API with Mayhem for API. Our CLI can also upload Mayhem for API results to GitHub Code Scanning from any CI.

GitHub Action

With our GiHub Action, you'll get Mayhem for API testing every API change in no time.

To integrate Mayhem for API into GitHub Actions workflows:

  1. Create a Service Account token for your organization
  2. Add the newly created token to your GitHub secrets.
  3. Create a workflow that starts your API & invoke the Mayhem for API action.

Configure your workflow

At the base directory of your code repository, add a .github/workflows/mapi.yml file to configure GitHub Actions to run Mayhem for API. Your file should look like this.

name: Mayhem for API
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  security:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2

    - name: Start your API
      run: ./run_your_api.sh &
      #           ^-------------- update this, but keep the '&' to run
      #                           the API in the background.

    - name: Run Mayhem for API to check for vulnerabilities
      uses: ForAllSecure/mapi-action@v1
      with:
        mapi-token: ${{ secrets.MAPI_TOKEN }}
        api-url: http://localhost:8000 # <- update this
        api-spec: your-openapi-spec-or-postman-collection.json # <- update this

This configuration tells GitHub Actions to run a workflow which checks out your code, starts your API, and then runs Mayhem for API against your API. We pass the Mayhem for API token from the GitHub secret, MAPI_TOKEN.

Here's a fully working example repository showing you an ideal integration that you can use as a template. For more details on Mayhem for API Action, visit the Mayem for API Action in the GitHub Actions marketplace.

GitHub Code Scanning

If you are testing a public repository, or a private repository on a GitHub Enterprise plan, we recommend generating a SARIF report. Uploading SARIF reports to GitHub allows you to see any issue found by Mayhem for API in the "Security" tab of your repository. In addition, if your API is sending back stacktraces as part of the error response body, Mayhem for API will show you exactly where in your code the issue happened, right in your PR diffs:

GitHub

Using GitHub action

To do so with our GitHub Action, after starting your API in the workflow, run the Mayhem for API action followed by github/codeql-action/upload-sarif@v1 to upload the report. Note continue-on-error needs to be set to true in the Mayhem for API action in order to run the next step, even if Mayhem for API finds issues.

- name: Run Mayhem for API to check for vulnerabilities
  uses: ForAllSecure/mapi-action@v1
  continue-on-error: true
  with:
    mapi-token: ${{ secrets.MAPI_TOKEN }}
    api-url: http://localhost:8000 # <- update this
    api-spec: your-openapi-spec-or-postman-collection.json # <- update this
    sarif-report: mapi.sarif

# Upload SARIF file (only available on public repos or github enterprise)
- name: Upload SARIF file
  uses: github/codeql-action/upload-sarif@v1
  with:
    sarif_file: mapi.sarif

Using our CLI

To upload our results to GitHub code scanning from anywhere, locally or from a CI, you can use our CLI. You'll need to pass a GitHub token with the security_event scope so that Mayhem for API can upload findings. See GitHub documentation on creating access tokens for more information. Once you have your token, simply run:

mapi run '<target>' '<duration>' '<spec>' --sarif 'mapi.sarif' --github-token '<github-token>'

Mayhem for API will attempt to infer the git information from the environment set by the CI, and the git repository on the filesystem. If those are not available, Mayhem for API will ask you to pass the missing information so that it can upload the findings to the right place.

In addition, Mayhem for API also supports GitHub Enterprise on premise! Just pass --github-api-url <your-github-api-url> to mapi run so that Mayhem for API knows where to upload the results. Your GitHub instance needs to be accessible from the host running our CLI, but it does not need to be accessible by our cloud infrastructure.

GitHub Application

Installing the Mayhem for API GitHub App into your Personal or Organization GitHub account provides additional benefits by automatically updating the checks for your builds and pull requests with the result of your API Fuzzer jobs.

Install the GitHub app into your Personal or Organization repository by following the direct link:

https://github.com/apps/mayhem-for-api

You must select which repositories you would like Mayhem for API to have access to. Once you have decided and authorized the app, we will attempt to automatically detect GitHub builds whenever you run the CLI as part of your build. We do this by matching your git repository commit hash/branch/remote with repositories which you have given our app access.

If we cannot determine your repository settings, you may set them manually when calling mapi run with additional options such as --branch, --remote and --sha. See mapi run --help for a full listing.

To see an example of the GitHub App in action, please visit our example repository:

https://github.com/ForAllSecure/api-fuzzer-swagger-petstore-demo

Best Practices & Troubleshooting

⚠️ Please do not attempt to fuzz any applications that are not owned or managed by you. The fuzz testing will look like an attack at worst and a probe at best to any organization that's not aware of what's happening. Please make sure to only fuzz your API assets. Thanks!

Here are some recommendations to maximize the value of fuzzing for your API:

  • Avoid running fuzzing against production services, as it can be HIGHLY disruptive. It generates a fair amount of load, and might delete resources you do not want deleted. We recommend using it on a QA or staging instance.

  • Populating the service with some realistic data will help the fuzzer uncover more issues.

  • Since the fuzzer may create many new entities in the service under state, resetting the state between runs will prevent the API from being bogged down by previous fuzzing state.

Troubleshooting

Looking at the job status using the CLI can help you understand what the fuzzer is doing, and if it's running into any issue.

  • The job logs will contain errors, if any. We aim to provide actionable feedback, making it obvious how to resolve the issue. For instance, the job log might contain messages about not being able to connect to the target API.

  • The endpoint coverage table can tell you a lot about what the fuzzer is able to exercise. Ideally you'd see all endpoints having 2xx response codes. If you don't, this means that the fuzzer could not generate a valid request. If you only see 401 unauthorized responses, then the fuzzer is having trouble with authentication. If you only see 400 response codes, that means the fuzzer could not generate a valid input for this endpoint, maybe because it was too complex to be generated by our fuzzer, or maybe because the specification did not match the behavior. If you're not sure why the fuzzer fails to generate 2xx responses, please reach out and we'll look into it and make the fuzzer better.

  • Sometimes, the job log will contain an error due to something on our end. Please feel free to reach out to mayhem4api@forallsecure.com if you run into one.

  • If you have a Swagger 2.0 specification, we will attempt to convert it to OpenAPI 3.0 for you. If this fails, you can also try using the swagger2openapi command line tool:

    npm install -g swagger2openapi
    swagger2openapi -p -y swagger-spec.json > openapi-spec.yaml
    
  • If you have a Postman 2.x specification, we will attempt to convert it to OpenAPI 3.0 for you. If this fails, you can also try using postman-to-openapi.

In addition, please feel free to reach out to us on Discord or to mayhem4api@forallsecure.com if you have any issues, feedback of feature requests. We are happy to help!

HTTP Proxy Configuration

If your target is behind a proxy, or you require a proxy to reach the API for Mayhem for API, you can provide proxy information for the mapi CLI. Support for common proxy environment variables follows:

Environment VariableDescription
HTTP_PROXYThe address to use when initiating HTTP connection(s)
HTTPS_PROXYThe address to use when initiating HTTPS connection(s)
NO_PROXYNetwork address(es)/range(s) and domains to exclude from proxying

Examples

Say you wish to capture all the communication between the fuzzer and your API. You can place a proxy between Mayhem for API and your API such as Charles Proxy (which listens on localhost:8888 by default) to intercept and record all traffic.

HTTP_PROXY=http://localhost:8888 mapi run <target> ... 

If your target is an https, you can use the HTTPS_PROXY variable:

HTTPS_PROXY=https://localhost:8888 mapi run <target> ... 

Proxying https traffic may result in some certificate errors when the CLI attempts to communicate with the API for Mayhem for API. This can be resolved by ignoring certain addresses from being sent to the proxy with the NO_PROXY variable. For example:

HTTPS_PROXY=https://localhost:8888 \
 NO_PROXY=mayhem4api.forallsecure.com \
 mapi run <target> ... 

The 'Mayhem for API' API

The Mayhem for API is available in OpenApi 3.0 format and can be interacted with using the online Swagger Editor.

Please note: this is an unstable, pre-release API under active development. If you want to know more, or if you want to build against it, we'd like to hear from you! Please get in touch by emailing us at mayhem4api@forallsecure.com.

Authentication

An API Token is required to access the 'Mayhem for API' API. The token must be presented as a Bearer token in the Authorization header with every request in order to access your resources.

For example, to fetch a list of API Targets directly from the API with curl with your API Token <YOUR_TOKEN>:

curl --request GET 'https://mayhem4api.forallsecure.com/api/v1/api-target' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_TOKEN>'

gRPC Fuzzing

Mayhem for API has experimental support for gRPC servers through the use of a gRPC gateway. The gateway comes with a protoc plugin. It reads a gRPC service definition and generates a reverse-proxy server which translates a RESTful JSON API into gRPC.

gRPC Proxy diagram

This guide will walk you through the steps to fuzz a gRPC server. While this method can be used with any language, the guide assumes you're using go. If you're not and are having trouble following the steps for another language, let us know and we'll be happy to help out.

Installing gRPC-Gateway

Follow the instructions in the README. You'll need to have go and protoc installed.

Generate Server Stub

The following command generates the proxy stubs from your .proto file:

protoc -I . --grpc-gateway_out . \
    --grpc-gateway_opt logtostderr=true \
    --grpc-gateway_opt paths=source_relative \
    --grpc-gateway_opt generate_unbound_methods=true \
    your/service/v1/your_service.proto

The command above generates the gateway stub at your/service/v1/your_service.pb.gw.go. We will import the stubs to write the proxy.

Write an entrypoint for the proxy

We will now write the proxy in rest-proxy.go. The proxy will listen on port 8081 and forward requests to your gRPC server.

package main

import (
  "context"
  "flag"
  "net/http"

  "github.com/golang/glog"
  "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
  "google.golang.org/grpc"

  gw "github.com/your-org/your-project/your/service/v1"
)

var (
  // command-line options:
  // gRPC server endpoint
  grpcServerEndpoint = flag.String("grpc-server-endpoint",  "localhost:10990", "gRPC server endpoint")
)

func run() error {
  ctx := context.Background()
  ctx, cancel := context.WithCancel(ctx)
  defer cancel()

  // Register gRPC server endpoint
  // Note: Make sure the gRPC server is running properly and accessible
  mux := runtime.NewServeMux()
  opts := []grpc.DialOption{grpc.WithInsecure()}
  err := gw.RegisterYourServiceHandlerFromEndpoint(ctx, mux,  *grpcServerEndpoint, opts)
  if err != nil {
    return err
  }

  // Start HTTP server (and proxy calls to gRPC server endpoint)
  return http.ListenAndServe(":8081", mux)
}

func main() {
  flag.Parse()
  defer glog.Flush()

  if err := run(); err != nil {
    glog.Fatal(err)
  }
}

You will need to change three things:

  • The import line "github.com/your-org/your-project/your/service/v1" to match your project and the path to the stub you generated in the previous step.
  • The url to your gRPC server. The code above assumes your gRPC server is listening on localhost:10990.
  • The call to gw.RegisterYourServiceHandlerFromEndpoint to match your service name. The function should be present in the gateway stub you generated earlier if you're not sure what the name should be.

Generate the OpenAPI Specification

The following command generates the OpenAPI specification:

protoc -I . --openapiv2_out . \
    --openapiv2_opt logtostderr=true \
    --openapiv2_opt generate_unbound_methods=true \
    your/service/v1/your_service.proto

Fuzz

  1. Start your gRPC server
  2. Start the proxy with go run rest-proxy.go
  3. Create a target: mapi target create target-name http://localhost:8081
  4. Fuzz: mapi run target-name 120 your/service/v1/your_service.openapi.json

Request Rewrite Plugins

Mayhem for API has experimental support for request rewriting via plugins, which are standaloane programs that implement a protobuf interface and expose it via gRPC.

The interface details are below. But first...

Do I Need This?

We've tried to make Mayhem for API work out of the box for as many use-cases as we can, and we'll keep adding more. The request rewrite plugin is intended as a kind of escape-hatch for situations where an API has requirements that we can't currently meet out of the box.

Here are the cases we had in mind when building the feature:

Just In Time Authentication

In any situation where the authentication needs to change during a fuzzing job, we don't currently have a solution directly in Mayhem for API.

A request rewrite plugin can inject the appropriate credentials at whatever cadence your API requires.

Request Digesting/Signing

If your API uses any kind of digesting or signing as authentication measures, Mayhem for API will be unable to generate valid requests.

A request rewrite plugin can compute and inject these per-request checksums or signatures.

Something else?

The request plugin interface is extremely open-ended, and can rewrite just about any part of any request.

If you're thinking about using a rewrite plugin for any reason, please get in touch. This is a very young, very raw feature, and we'd like to know how people are using it, and how we can make it better!

Interface

Invocation

To add a request rewrite plugin to a fuzzing run, use the --experimental-rewrite-plugin <path-to-plugin-executable> argument to mapi run.

Executable gRPC Server

mapi will run the executable with no command-line arguments, and expects it to be a long-lived process that:

  • listens for gRPC traffic on a localhost port
  • prints the port number, followed by a newline, to stdout as the first line

Beyond that, it can do whatever it wants. Additional stdout/stderr output from the plugin are forwarded to the mapi debug logs.

At the end of the fuzzing run, mapi will attempt to kill the plugin process.

Protobuf

Request rewrite plugins implement the following protobuf interface:

syntax = "proto3";

package mapi.rewrite;

service RewritePlugin {
   rpc Rewrite(Request) returns (Request);
}

message Request {
   message Header {
      bytes name = 1;
      bytes value = 2;
   }
   string url = 1;
   repeated Header headers = 2;
   bytes body = 3;
}

Example

Here's an example request rewrite plugin, which does... absolutely nothing, just returning the original request unchanged!

First, generate all of the gRPC and protobuf boilerplate from the .proto above:

$ pip install grpcio-tools
$ python -m grpc_tools.protoc --python_out=. --grpc_python_out=. -I. request-rewrite-plugin.proto

Then create a python program which implements the RewritePluginServicer, stands up a gRPC server, and lets mapi know what port it's listening on:

#!/usr/bin/env python

import sys
from concurrent import futures
import grpc
import request_rewrite_plugin_pb2
import request_rewrite_plugin_pb2_grpc

## implement the RewritePlugin interface
class RewritePluginServicer(request_rewrite_plugin_pb2_grpc.RewritePluginServicer):
    def Rewrite(self, request, context):
        ## well, this isn't very interesting :)
        return request

if __name__ == '__main__':
    ## boot up the gRPC server
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    request_rewrite_plugin_pb2_grpc.add_RewritePluginServicer_to_server(
        RewritePluginServicer(), server)
    server.add_insecure_port('127.0.0.1:50051')
    server.start()
    
    ## inform mapi of the port we're listening on
    print("50051")
    sys.stdout.flush()

    server.wait_for_termination()

Make sure your .py file is executable, then pass it to mapi via --experimental-rewrite-plugin. That's it!

Caveats and Cautions

bytes, Not string!

Mayhem for API requests are not always representable as strings. Be careful!

Subject to Change

All of this is volatile and subject to change. If you're using this feature, please get in touch, so that we can keep you in the loop as we consider refinements and changes.

Response Classify Plugins

Mayhem for API has experimental support for response classification via plugins, which are standaloane programs that implement a protobuf interface and expose it via gRPC.

The interface details are below. But first...

Do I Need This?

We've tried to make Mayhem for API work out of the box for as many use-cases as we can, and we'll keep adding more. These plugins are intended as a kind of escape-hatch for situations where an API has requirements that we can't currently meet out of the box.

Here are the cases we had in mind when building the feature:

Sensitive Data Leaking

If there are certain keys, values or patterns that you want to assert should never show up in your API output, regardless of inputs, Mayhem for API doesn't currently have a direct solution.

A response classify plugin can make these assertions.

Nonstandard Response Code Assertions

The rules Mayhem for API follows based on each endpoint's response codes are baked in, and based on standards. There may be specific endpoints where you want to apply more strict constraints—for example, an endpoint whose contract is that it never returns a 400 error.

A response classify plugin can be used to check for this kind of constraint.

Something else?

The classifier plugin interface is extremely open-ended, and can be used to check just about any aspect of a response.

If you're thinking about using a classify plugin for any reason, please get in touch. This is a very young, very raw feature, and we'd like to know how people are using it, and how we can make it better!

Interface

Invocation

To add a response classify plugin to a fuzzing run, use the --experimental-classify-plugin <path-to-plugin-executable> argument to mapi run.

Executable gRPC Server

mapi will run the executable with no command-line arguments, and expects it to be a long-lived process that:

  • listens for gRPC traffic on a localhost port
  • prints the port number, followed by a newline, to stdout as the first line

Beyond that, it can do whatever it wants. Additional stdout/stderr output from the plugin are forwarded to the mapi debug logs.

At the end of the fuzzing run, mapi will attempt to kill the plugin process.

Protobuf

Response classify plugins implement the following protobuf interface:

syntax = "proto3";

package mapi.classify;

service ClassifyPlugin {
   rpc Classify(Response) returns (Issues);
}

message Response {
   message Header {
      bytes name = 1;
      bytes value = 2;
   }
   string request_url = 1;
   string final_url = 2;
   uint32 status = 3;
   repeated Header headers = 4;
   bytes body = 5;
}

message Issues {
   message Issue {
      string summary = 1;
   }
   repeated Issue issues = 1;
}

Example

Here's an example classifier plugin, which does... absolutely nothing, just acknowledging each response without flagging any issues.

First, generate all of the gRPC and protobuf boilerplate from the .proto above:

$ pip install grpcio-tools
$ python -m grpc_tools.protoc --python_out=. --grpc_python_out=. -I. response-classify-plugin.proto

Then create a python program which implements the ClassifyPluginServicer, stands up a gRPC server, and lets mapi know what port it's listening on:

#!/usr/bin/env python

import sys
from concurrent import futures
import grpc
import response_classify_plugin_pb2
import response_classify_plugin_pb2_grpc

## implement the ClassifyPlugin interface
class ClassifyPluginServicer(response_classify_plugin_pb2_grpc.ClassifyPluginServicer):
    def Classify(self, response, context):
        ## well, this isn't very interesting :)
        issues = response_classify_plugin_pb2.Issues()
        return issues

if __name__ == '__main__':
    ## boot up the gRPC server
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    response_classify_plugin_pb2_grpc.add_ClassifyPluginServicer_to_server(
        ClassifyPluginServicer(), server)
    server.add_insecure_port('127.0.0.1:50051')
    server.start()

    ## inform mapi of the port we're listening on
    print("50051")
    sys.stdout.flush()

    server.wait_for_termination()

Make sure your .py file is executable, then pass it to mapi via --experimental-classify-plugin. That's it!

Caveats and Cautions

bytes, Not string!

API responses are not always representable as strings. Be careful!

Subject to Change

All of this is volatile and subject to change. If you're using this feature, please get in touch, so that we can keep you in the loop as we consider refinements and changes.

Open Source Notices

Mayhem For API ❤️ open-source software! We rely on a number of open-source dependencies, and we are grateful for the work all the maintainers are putting into them. To give back, we regularly upstream any improvements we make.

Here are the copyright notices of our CLI dependencies:

Here are the copyright notices of our Frontend dependencies:

Release Notes

Supported Versions

2.7.9

  • CLI - Added mapi job delete subcommand

2.7.8

  • CLI - Fixed generated SARIF when parsed stacktraces reference "line 0"
  • CLI - Added ability to filter fuzzed endpoints by their OpenAPI tags

2.7.7

  • CLI - Expanded response spec validation warnings
  • CLI - Fixed an issue with failing to stop a run against an unreachable API

2.7.6

  • CLI - Added experimental support for response classifying plugins, which can be used to implement arbitrary logic for identifying issues

2.7.3

  • CLI - If present, OpenAPI tags will be used to organize and group issues
  • CLI - Fixed an issue with authentication redaction in issue details

2.7.2

  • Invalid specification issues will now be reported with "warning" severity if API responses do not conform with the API specification.
  • SARIF/JUnit Report - Fixed an issue where an extra '/' prefix may appear in path display

2.7.1

  • CLI - Support suppressing issues - see Suppressing Issues for more details.
  • Checkers - Added support for Error-based NoSQL Injection for MongoDB

2.7.0

  • CLI - mapi will report warnings when responses are incompatible with the specification used to fuzz your target
  • CLI - Fixed some issues with C# stacktrace parsing
  • CLI/SARIF - It is now possible to report multiple issues for the same path, method and rule id. This is most effective when fuzzing a target that is running in debug mode and will respond with a stacktrace/traceback on error.
    • For example, a 500 Internal Server Error triggered on GET /foo, resulting in different stacktraces will be reported as two separate issues:
      • Issue 1: GET /foo?bar=1 -> Internal Server Error (NullPointerException)
      • Issue 2: GET /foo?bar=0 -> Internal Server Error (DivideByZeroException)

2.6.18

  • CLI - Save fuzzing run output to an HTTP Archive (.har file) with mapi run ... --har out.har

2.6.16

  • CLI - Added beta support for fuzzing with an HTTP Archive (.har file) for the specification.

    • mapi run will now accept a .har file for the specification argument
    • It is recommended to convert a .har to an OpenAPI specification with mapi convert har... to inspect the converted specification before running a job
  • CLI - Added mapi convert command. This runs automatically when calling mapi run with a document which supports conversion.

    The convert subcommand provides an opportunity to inspect the converted OpenAPI 3.0 document prior to fuzzing an API.

    • Convert HTTP Archive .har file to an OpenAPI spec with mapi convert har <recording.har>
    • Convert Swagger 2.0 json or yaml file to an OpenAPI spec with mapi convert swagger2 <spec.json|yaml>
    • Convert Postman 2.x collection json or yaml file to an OpenAPI spec with mapi convert postman <collection.json|yaml>
    • Run mapi convert --help for more details

2.6.14

  • CLI - Issues from the most recent previous job will now be replayed at the end of a run. The intent of this change is to reduce issues fluctuating between jobs.
    • To opt-out of this behavior, pass the --no-replay flag to mapi run.

2.6.13

  • The CLI now handles specifications with missing path parameters. If a path parameter is not in the specification, Mayhem for API will now infer it automatically and fuzz it. Previously, the CLI would exit early with an error.

2.6.12

  • CLI - Resolve an issue where the CLI may panic when generating some invalid URLs during a fuzzing run
  • Fixed an issue where calling mapi run with a path to the specification on Windows (C:\path\to\spec.yaml) would fail to read specification

2.6.11

  • CLI - Resolved an issue that could cause duplicate reported issues in certain corner conditions
    • Issue replay now supports providing a modified OpenAPI/Swagger/Postman specification
    • MAPI_LOG environment variable will now correctly output trace and debug level logs

2.6.10

  • CLI: Support for issue replay. Replay issues using mapi issue replay <job_id>
    • Try mapi issue replay --help to see more details
    • Note that some issues depend on the state of your API and may not always reproduce

2.6.9

  • CLI: Show incomplete request counts in interactive job display
  • CLI: Add markdown display support to SARIF output
  • Header parameters listed in API specifications will now be fuzzed

2.6.7

  • New Issue: Server Crash
    • Server crashes are now detected and reported in Job Issues.
    • A server that can be forced to crash may be vulnerable to denial-of-service attacks.
  • CLI: mapi issue list print a list of issues for a job

2.6.5

  • Fix parsing of stacktraces generated on windows host.

2.6.4

  • Authentication data is now redacted from buggy requests before being uploaded to our API. Note that in some cases, slightly mutated auth tokens might still be updated to our API (since they might be necessary to demonstrate authentication bypasses)
  • Add support for C# stacktraces with async code and nested exceptions

2.6.3

  • Legacy github token (which are not prefixed by ghp_) are now accepted.

2.6.0

  • Enable new security checkers by default: Command injection, Blind command injection, Path traversals, SSRF (assumes CLI can be reached through localhost from the API)
  • Improve authentication bypass checker by differentiating between -H (for regular headers) and --header-auth (for authentication header).
  • Redirect loops will not terminate a run anymore
  • Improvements in the fuzzing engine to increase coverage

2.5.16

  • CLI: Ignore errors with the --ignore-error option to mapi run. Callers may ignore specific errors such as InternalServerError or AuthenticationBypass when including this flag.

2.5.13

  • CLI: Upload code scanning results to GitHub from anywhere by passing --github-token with a token to mapi run.
  • CLI: Upload code scanning results to your on-premise GitHub instance by passing --github-api-url to mapi run.
  • CLI: Auto-detect build information from the environment in Drone CI
  • CLI: Golang stacktrace parsing, in addition to the existing parsing for Ruby, JS, Python and Java.

2.5.10

  • CLI: Add SARIF support. mapi run now accepts a --sarif <output-file> option which outputs a SARIF file that you can upload to supported systems, like GitHub if you have enterprise plan.

2.5.7

  • The CLI now has beta support for postman 2.x collections 🚀! You can pass your collection to mapi run instead of an OpenAPI specifications.

2.5.6

  • CLI: mapi run can now automatically create a new API Target if the --url option is supplied.

2.5.5

  • Security: Added support for detecting authentication configuration issues. The fuzzer will now periodically remove or mangle any authentication parameters used for a job. Any endpoints that are specified as with security configuration will be marked as buggy if they can be accessed successfully with missing or altered authentication parameters.

2.5.4

  • Security: Added support for detecting SQL Injection errors. See the new documentation on configuring your API for tips on improving detection of SQL Injection problems.

  • CLI: Change the logging level of the mapi CLI with the MAPI_LOG environment variable. Supported values include trace,debug,info,warn,error. The default is set to info.

2.4.1

  • Renamed cohorts to organizations

2.3.28

  • Added experimental path traversal detection, enabled by setting the MAPI_FEATURE_PATH_TRAVERSAL environment variable.

2.3.19

  • Added support to CLI for rending email verification with mapi signup resend-verify <email-address>

2.3.18

  • Creating a new account when joining a Cohort now requires email verification
  • Added support to CLI for completing email verification with mapi signup verify

2.3.16

  • Improve parsing of URLs given through --url
  • Fix memory leak in type generation during fuzzing

2.3.15

  • Renamed the product to 'Mayhem for API' and the CLI to 'mapi'. Please re-download the CLI.

2.3.8

  • Parallel fuzzing is now available using -j when invoking mapi run (defaults to 1). This will allow you to potentially find concurrency issues, and load test your APIs.

2.3.7

  • bug fix: better support for reference loops in specifications

2.3.5

  • mapi run will now generate a human-friendly html report when --html is given.

2.3.4

  • CLI prints warnings on unexpected properties in the specification instead of aborting.

2.3.3

  • API Targets will now include their owner when presented in the CLI or when referenced to start a new run.

    You may continue to refer to any existing API Targets without the owner; however, shared API Targets will require the owner to be specified.

2.3.2

  • Windows support: Our CLI can now run natively on 64-bit windows.

2.3.0

  • Users can specify cookies for the fuzzer to authenticate to the target API.

2.2.4

  • EULA agreement

2.2.3

  • GitHub Cloud Integration support is now available. Please reach out to us at mayhem4api@forallsecure.com so that you can get connected with GitHub. Once you have been approved, you can connect with the CLI with the command:

    mapi github connect
    
  • Our GitHub app is also live. During Limited Availability it will not be accessible via the marketplace, but can be installed into your Personal or Organization account by following this link:

    https://github.com/apps/mayhem-for-api

2.2.2

  • Preparation for GitHub support. The CLI will attempt to gather CI details from GitHub Actions, Jenkins, or simply invocations of git to synchronize Job Status with GitHub checks. If details cannot be inferred from the environment, then they may be provided manually with new optional arguments passed to mapi run.
  • GitHub integration support will be fully supported once the Mayhem for API app is available in the GitHub marketplace.

2.2.1

  • CLI will auto-update across patch and minor versions

2.2.0

  • CLI supports API pagination for job and target API resources

2.1.10

  • Fix unresponsive arrow keys in interactive job status

2.1.9

  • JUnit output contains percentile response times per endpoint.
  • Fuzzer better uses examples in the API specification, which allows users to give a hint to the fuzzer when it cannot generate 200 responses on its own.
  • Fuzzer warns if the API closes a connection early before sending a full response. This will eventually be classified as a bug.

2.1.8

  • For accounts registered via GitHub, a New Token may now be requested when using mapi github connect --new-token.

2.1.7

  • Added interactive mode to mapi run with the new --interactive flag. This will now open the interactive UI that was previously only accessible with the mapi job status sub-command.
  • Job status UI now detects when a tty is not present. In instances where a tty is not present, the job log and coverage table will be printed to standard output.
  • All covered endpoints and statistics will now print to the console at the end of a run

2.1.6

  • Bug fixes for GitHub connect

2.1.5

  • Fuzzer will not consider HTTP response code 501 as an error
  • CLI will exit with return code 2 on fuzzer error
  • CLI will now automatically detect Swagger 2.0 specifications and attempt to convert them to OpenAPI 3.0 automatically with the help of the openapi-webconverter

2.1.4

  • CLI can now override API Target authentication parameters when calling mapi run

2.1.3

  • CLI support for handling upcoming pagination changes to job and target API resources

Breaking Changes

As Mayhem for API is continuously evolving, we make an effort to avoid breaking changes. Not breaking your workflow and CI builds is of utmost important to us. We want it to just work, always.

There are times due to security constraints or technical limitations that require us to make a breaking change. We aim to make this as painless as possible, usually requiring no more than downloading a newer CLI. If breaking changes are required, we will try to make it as painless as possible:

  • We will give you a heads up if we notice you are using a version that will become incompatible.
  • We aim to provide command-line arguments backwards compatibility, so that you can always use the latest release without having to change your invocation.
  • We give you tips on how to integrate the fuzzer into your builds so that you automatically use new releases.
  • When necessary, we will slowly deprecate features or workflows, giving you enough time to upgrade.

Unsupported versions

2.1.2

  • CLI includes request and response in junit failures

2.1.1

  • CLI support for blacklisting endpoints when calling mapi run

2.1.0

  • CLI improved logging, feedback and error handling
  • Fixed cascade delete for API Targets

Current Terms and Policies