We recommend running Mayhem for API as part of your CI pipelines. Adding the fuzzer should take only a few minutes. Running the fuzzer as part of your continuous build will automatically check your API with your latest changes before they are merged and deployed.
Downloading the CLI
The first step is to download the CLI. We recommend using the
latest url to
always get the latest release. We aim to retain backwards compatibility for
command line arguments so that your build always works.
Assuming your build is running in Docker, we recommend adding the following lines to add the CLI to your Docker image:
ADD https://mayhem4api.forallsecure.com/downloads/cli/latest/linux-musl/mapi /usr/bin/mapi RUN chmod 0755 /usr/bin/mapi
This ensures that you get the latest release every time, without affecting the Docker build cache for follow on layers if there hasn't been a new release. This will download the CLI (~25MB) every time you build the image. We recommend adding the line near the end of your build, shortly before you add your code, to minimize the number of uncached layers to rebuild when there is a new CLI release.
Optionally, to avoid re-downloading the CLI every time, we recommend using Docker's BuildKit. BuildKit checks the URL to see if a new version is available, and only downloads it if necessary. This ensures you get the latest release at all times without slowing down your builds with unnecessary downloads. Enable BuildKit with a simple environment variable:
$ DOCKER_BUILDKIT=1 docker build .
Finally, if you happen to use Docker's experimental syntax, you can download the CLI in one command:
ADD --chmod=755 https://mayhem4api.forallsecure.com/downloads/cli/latest/macos/mapi /usr/bin/mapi
The CLI requires a token to function. We recommend storing your mapi token in
your CI secret stores, and loading it in the
variable. The CLI will use the value in that environment variable
Running the fuzzer
To fuzz your API, you'll have to first start your API locally following our best practices. Once it's up and running, invoke the fuzzer with:
mapi run <target> <duration> <spec-file> --url <url> --junit results.xml
<target>is the name of your api target that you created with
mapi target create. The target name is often the name of your project.
<duration>depends on how long you want your build to take. We recommend
auto, but you may specify a fixed duration such as
60secinstead. To maximize fuzzing duration without slowing down the build, we recommend fuzzing in parallel to other build steps, for the duration of those build steps.
- If your API requires authentication for some endpoints, consider passing it to the fuzzer for best results.
--junit options are optional. If
unspecified, it will come from the URL you specified in the API target.
--junit is only necessary if you want your CI to parse a JUnit XML to extract
the tests and potential failures (see below).
Gathering the results
The CLI will return an error code if any 500 responses are observed. This can
be used to fail the build if you so choose. In addition, with the
option, the fuzzer will output a JUnit XML file that your CI can parse.
The JUnit will contain a sample request/response for any endpoint that returned a 500, which can be helpful to debug the issue (especially if the API is run in debug mode and the response contains debugging output like stacktraces).
To get a human-friendly report that you can save as a build artifact, use the
to get a report like the following:
You may filter out certain types of response errors that you do not want to appear in
your results using the CLI. For example to prevent 500 Internal Server Error responses
from being marked as issues, include the
--ignore-rule InternalServerError flag in your
--ignore-rule flag may be used multiple times to ignore more than one type
of error. By default, ALL detected response errors will be classified as issues.
mapi run <target> <duration> <url> --ignore-rule InternalServerError
💡 To see a full list of
--ignore-ruleoptions, see the output of
mapi run --help.