CI Integration

We recommend running Mayhem for API as part of your CI pipelines. Adding the fuzzer should take only a few minutes. Running the fuzzer as part of your continuous build will automatically check your API with your latest changes before they are merged and deployed.

Downloading the CLI

The first step is to download the CLI. We recommend using the latest url to always get the latest release. We aim to retain backwards compatibility for command line arguments so that your build always works.

Assuming your build is running in Docker, we recommend adding the following lines to add the CLI to your Docker image:

ADD https://mayhem4api.forallsecure.com/downloads/cli/latest/linux-musl/mapi /usr/bin/mapi
RUN chmod 0755 /usr/bin/mapi

This ensures that you get the latest release every time, without affecting the Docker build cache for follow on layers if there hasn't been a new release. This will download the CLI (~25MB) every time you build the image. We recommend adding the line near the end of your build, shortly before you add your code, to minimize the number of uncached layers to rebuild when there is a new CLI release.

Optionally, to avoid re-downloading the CLI every time, we recommend using Docker's BuildKit. BuildKit checks the URL to see if a new version is available, and only downloads it if necessary. This ensures you get the latest release at all times without slowing down your builds with unnecessary downloads. Enable BuildKit with a simple environment variable:

$ DOCKER_BUILDKIT=1 docker build .

Finally, if you happen to use Docker's experimental syntax, you can download the CLI in one command:

ADD --chmod=755 https://mayhem4api.forallsecure.com/downloads/cli/latest/macos/mapi /usr/bin/mapi

Authentication

The CLI requires a token to function. We recommend storing your mapi token in your CI secret stores, and loading it in the MAPI_TOKEN environment variable. The CLI will use the value in that environment variable automatically.

Running the fuzzer

To fuzz your API, you'll have to first start your API locally following our best practices. Once it's up and running, invoke the fuzzer with:

mapi run <target> <duration> <spec-file> --url <url> --junit results.xml

• <target> is the name of your api target that you created with mapi target create. The target name is often the name of your project.
• <duration> depends on how long you want your build to take. We recommend auto, but you may specify a fixed duration such as 60sec instead. To maximize fuzzing duration without slowing down the build, we recommend fuzzing in parallel to other build steps, for the duration of those build steps.
• If your API requires authentication for some endpoints, consider passing it to the fuzzer for best results.

Note that --url and --junit options are optional. If --url is unspecified, it will come from the URL you specified in the API target. --junit is only necessary if you want your CI to parse a JUnit XML to extract the tests and potential failures (see below).

Gathering the results

The CLI will return an error code if any 500 responses are observed. This can be used to fail the build if you so choose. In addition, with the --junit option, the fuzzer will output a JUnit XML file that your CI can parse.

The JUnit will contain a sample request/response for any endpoint that returned a 500, which can be helpful to debug the issue (especially if the API is run in debug mode and the response contains debugging output like stacktraces).

To get a human-friendly report that you can save as a build artifact, use the --html option to get a report like the following:

Ignoring Errors

You may filter out certain types of response errors that you do not want to appear in your results using the CLI. For example to prevent 500 Internal Server Error responses from being marked as issues, include the --ignore-rule InternalServerError flag in your call to mapi run.

The --ignore-rule flag may be used multiple times to ignore more than one type of error. By default, ALL detected response errors will be classified as issues.

mapi run <target> <duration> <url> --ignore-rule InternalServerError

💡 To see a full list of --ignore-rule options, see the output of mapi run --help.