Script Reference - Artillery - Io Docs Payload
Script Reference - Artillery - Io Docs Payload
The config section defines the target (the hostname or IP address of the system under test), the load
progression, and protocol-specific settings such as HTTP response timeouts or Socket.io transport options.
The scenarios section contains definitions for one or more scenarios for the virtual users (VUs) that
Artillery will create.
For example, to set a default HTTP header for all requests via the SERVICE_API_KEY environment variable,
your test definition would look like this:
config:
target: https://my.microservice.internal
phases:
- duration: 600
arrivalRate: 10
defaults:
headers:
x-api-key: "{{ $processEnvironment.SERVICE_API_KEY }}"
scenarios:
- flow:
- get:
url: "/"
Which would allow for the API key to be left out of the source code and provided on-the-fly with:
export SERVICE_API_KEY="012345-my-api-key"
artillery run my-test.yaml
Load Phases
A load phase defines how many new virtual users (VUs) will be generated in a time period. For example, a
typical performance test will have a gentle warm up phase, followed by a ramp up phase which is then
followed by a maximum load for a duration of time.
config.phases is an array of phase definitions that Artillery goes through sequentially. Four kinds of
phases are supported:
A phase with a duration and a constant arrival rate of a number of new VUs per second.
A linear ramp up phase where the number of new arrivals increases linearly over the duration of the
phase.
A phase which generates a fixed count of new arrivals over a period of time.
A pause phase which generates no new VUs for a duration of time.
This is important so we're going to mention it again. The duration of an arrival phase determines only
how long virtual users will be generated for. It is not the same as the duration of a test run. How long
a given test will run for depends on several factors, such as complexity and length of user scenarios,
server response time, and network latency.
Examples
Create 50 virtual users every second for 5 minutes:
CONSTANT ARRIVAL RATE
config:
target: "https://staging.example.com"
phases:
- duration: 300
arrivalRate: 50
config:
target: "https://staging.example.com"
phases:
- duration: 300
arrivalRate: 10
maxVusers: 50
Ramp up arrival rate from 10 to 50 over 2 minutes, followed by 10 minutes at 50 arrivals per second.
config:
target: "https://staging.example.com"
phases:
- duration: 120
arrivalRate: 10
rampTo: 50
name: "Warm up the application"
- duration: 600
arrivalRate: 50
name: "Sustained max load"
(The name attributes are optional. They are useful to help identify load phases in the reports generated by
Artillery.)
FIXED COUNT OF ARRIVALS
config:
target: "https://staging.example.com"
phases:
- duration: 60
arrivalCount: 20
phases:
- duration: 100
arrivalRate: 0
rampTo: 50
is equivalent to:
phases:
-
arrivalRate: 0
duration: 1.96
-
arrivalRate: 1
duration: 1.96
-
arrivalRate: 2
duration: 1.96
-
... etc ...
-
arrivalRate: 50
duration: 2
Environments
Typically we'd want to re-use a load testing script across multiple enviroments (e.g. dev , staging , and
production ) with minor tweaks. That's what config.environments is for: a number of named
environments can be defined with environment-specific configuration.
For example, a typical use-case is to define multiple targets with di erent load phase definitions for each of
those:
config:
target: "http://wontresolve.local:3003"
phases:
- duration: 10
arrivalRate: 1
environments:
production:
target: "http://wontresolve.prod:44321"
phases:
- duration: 120
arrivalRate: 10
staging:
target: "http://127.0.0.1:3003"
phases:
- duration: 1200
arrivalRate: 20
scenarios:
- ...
config:
#
# config here
#
scenarios:
- flow:
- log: "Current environment is set to: {{ $environment }}"
Payload Files
It can often be useful to be able to inject data from external files into your test scenarios. For example, you
might have a list of usernames and passwords that you want to use to test the auth endpoint in your API.
Payload files are in the CSV format and Artillery allows you to map each of the rows to a variable name that
can be used in scenario definitions. For example:
config:
payload:
# path is relative to the location of the test script
path: "users.csv"
fields:
- "username"
- "password"
scenarios:
- flow:
- post:
url: "/auth"
json:
username: "{{ username }}"
password: "{{ password }}"
We tell Artillery to load users.csv file and make variables username and password available in
scenarios containing values from one of the rows in the CSV file
payload:
-
path: "./pets.csv"
fields:
- "species"
- "name"
-
path: "./urls.csv"
fields:
- "url"
It's also possible to load a di erent CSV file depending on the environment you set with the
--environment flag. For example, you may want to load a di erent set of usernames/passwords to use
with an authentication endpoint when running the same test in di erent environments:
payload:
- path: "{{ $environment }}-logins.csv"
fields:
- "username"
- "password"
Options
order (default: random ) -- control how rows are picked from the CSV file for each new virtual user.
Set to sequence to iterate through the rows in a sequence (looping around and starting from the
beginning after the last row has been reached).
skipHeader (default: false ) -- set to true to make Artillery skip the first row in the file (i.e. the
header row)
delimiter (default: , ) -- if a delimiter other than comma is used, set this option to the delimiter
character
cast (default: true ) -- by default Artillery will convert fields to native types (e.g. numbers or
booleans). To keep those as string, set this option to false .
skipEmptyLines -- by default Artillery skips empty lines in the file. Set to false to include empty
lines.
Example
config:
payload:
path: "users.csv"
fields:
- "username"
- "password"
order: sequence
skipHeader: true
scenarios:
- # ... the rest of the script
Inline variables
Variables can defined in the config.variables section of a script and used in subsequent request
templates.
Variables defined in this block are only available in scenario definitions, i.e. they cannot be used to
template any values in the `config` section of your scripts. If you need to dynamically override values in
the `config` section, use environment variables in conjunction with $processEnvironment .
config:
target: "http://app01.local.dev"
phases:
-
duration: 300
arrivalRate: 25
variables:
postcode:
- "SE1"
- "EC1"
- "E8"
- "WH9"
id:
- "8731"
- "9965"
- "2806"
The variables can then be used in templates as normal. For example: {{ id }} and {{ postcode }} .
Latency
To check that the aggregate p95 latency is 200ms or less, and make Artillery exit with a non-zero exit code
if it's over 200, add the following configuration to your script:
config:
ensure:
p95: 200
Error rate
The error rate is defined as the ratio of virtual users that didn't complete their scenarios successfully to the
total number of virtual users created during the test.
To make Artillery exit with a non-zero if the total error rate exceeded 1%:
config:
ensure:
maxErrorRate: 1
scenarios
The scenarios section is where one or more virtual user scenarios are defined.
A scenario is a sequence of steps that will be run sequentially which represents a typical sequence of
requests or messages sent by a user of an application.
A scenario definiton is an object which must contain a flow attribute and may contain a number of other
attributes.
flow - a "flow" is an array of operations that a virtual user performs, e.g. GET and POST requests for
an HTTP (/docs/http-reference)-based application.
name - allows to assign a descriptive name to a scenario, e.g.
"search for a product and get its details"
weight - allows for the probability of a scenario being picked by a new virtual user to be "weighed"
relative to other scenarios
Some Artillery engines will also support other scenario attributes, e.g. the HTTP engine (/docs/http-
reference) allows for scenario-level hooks to be defined.
Scenario weights
Weights allow you to specify that some scenarios should be picked more often than others. If you have
three scenarios with weights 1 , 2 , and 5 , the scenario with the weight of 2 is twice as likely to be picked
as the one with the weight of 1 , and 2.5 times less likely than the one with weight of 5 . Or in terms of
probabilities:
Weights are optional, and if not specified are set to 1 (i.e. each scenario is equally likely to be picked).