0% found this document useful (0 votes)
305 views

Script Reference - Artillery - Io Docs Payload

The document summarizes key aspects of Artillery test scripts, including: - Artillery test scripts have two sections: config and scenarios. The config section defines test parameters like the target, load phases, and protocol settings. The scenarios section defines individual test scenarios. - Load phases in the config section define how virtual users are ramped up over time. Common phase types include constant arrival, linear ramp, fixed count, and pause. - Variables and payload files can be used to inject dynamic values into test scenarios from environment variables, inline definitions, and CSV files. - Configurations and variables can be defined separately for different environments to facilitate test reuse across environments.

Uploaded by

kjnjkn
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
305 views

Script Reference - Artillery - Io Docs Payload

The document summarizes key aspects of Artillery test scripts, including: - Artillery test scripts have two sections: config and scenarios. The config section defines test parameters like the target, load phases, and protocol settings. The scenarios section defines individual test scenarios. - Load phases in the config section define how virtual users are ramped up over time. Common phase types include constant arrival, linear ramp, fixed count, and pause. - Variables and payload files can be used to inject dynamic values into test scenarios from environment variables, inline definitions, and CSV files. - Configurations and variables can be defined separately for different environments to facilitate test reuse across environments.

Uploaded by

kjnjkn
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Script Reference

An Artillery test script is composed of two sections: config and scenarios .

The config section defines the target (the hostname or IP address of the system under test), the load
progression, and protocol-specific settings such as HTTP response timeouts or Socket.io transport options.

The scenarios section contains definitions for one or more scenarios for the virtual users (VUs) that
Artillery will create.

The config Section


target - the URI of the application under test. For an HTTP application it's the base URL for all
requests, e.g. http://myapp.staging.local . For a WebSocket server it would be the hostname
(and optionally the port) of the server, e.g. ws://127.0.0.1 .
environments - specify a list of environments, and associated target URLs; see below
phases - specify the duration of the test and frequency of requests; see below
payload - used for importing variables from a CSV file; see below
variables - set variables inline rather than loading them from an external CSV file
defaults - set default headers that will apply to all HTTP requests
tls - configure how Artillery handles self-signed certificates. See HTTP Reference (/docs/http-
reference)
ensure - set success conditions for latency or error rates; useful for CI/CD

Using dynamic values in config


Values can be set dynamically via environment variables which are available under $processEnvironment
template variable.

For example, to set a default HTTP header for all requests via the SERVICE_API_KEY environment variable,
your test definition would look like this:
config:
target: https://my.microservice.internal
phases:
- duration: 600
arrivalRate: 10
defaults:
headers:
x-api-key: "{{ $processEnvironment.SERVICE_API_KEY }}"
scenarios:
- flow:
- get:
url: "/"

Which would allow for the API key to be left out of the source code and provided on-the-fly with:

export SERVICE_API_KEY="012345-my-api-key"
artillery run my-test.yaml

Load Phases
A load phase defines how many new virtual users (VUs) will be generated in a time period. For example, a
typical performance test will have a gentle warm up phase, followed by a ramp up phase which is then
followed by a maximum load for a duration of time.

config.phases is an array of phase definitions that Artillery goes through sequentially. Four kinds of
phases are supported:

A phase with a duration and a constant arrival rate of a number of new VUs per second.
A linear ramp up phase where the number of new arrivals increases linearly over the duration of the
phase.
A phase which generates a fixed count of new arrivals over a period of time.
A pause phase which generates no new VUs for a duration of time.

Total number of VUs can be capped with the maxVusers option.

 This is important so we're going to mention it again. The duration of an arrival phase determines only
how long virtual users will be generated for. It is not the same as the duration of a test run. How long
a given test will run for depends on several factors, such as complexity and length of user scenarios,
server response time, and network latency.

Examples
Create 50 virtual users every second for 5 minutes:
CONSTANT ARRIVAL RATE
config:
target: "https://staging.example.com"
phases:
- duration: 300
arrivalRate: 50

Constant arrival rate with no more than 50 concurrent VUs

config:
target: "https://staging.example.com"
phases:
- duration: 300
arrivalRate: 10
maxVusers: 50

A RAMP UP FOLLOWED BY AN CONSTANT ARRIVAL RATE

Ramp up arrival rate from 10 to 50 over 2 minutes, followed by 10 minutes at 50 arrivals per second.

config:
target: "https://staging.example.com"
phases:
- duration: 120
arrivalRate: 10
rampTo: 50
name: "Warm up the application"
- duration: 600
arrivalRate: 50
name: "Sustained max load"

(The name attributes are optional. They are useful to help identify load phases in the reports generated by
Artillery.)
FIXED COUNT OF ARRIVALS

Create 20 virtual users in 60 seconds (approximately one every 3 seconds):

config:
target: "https://staging.example.com"
phases:
- duration: 60
arrivalCount: 20

A DO-NOTHING PAUSE PHASE


config:
target: "https://staging.example.com"
phases:
- pause: 60

How do ramps work?


Think of a rampTo as a shortcut for manually writing out a sequence of arrival phases, e.g. the following
ramp:

phases:
- duration: 100
arrivalRate: 0
rampTo: 50

is equivalent to:

phases:
-
arrivalRate: 0
duration: 1.96
-
arrivalRate: 1
duration: 1.96
-
arrivalRate: 2
duration: 1.96
-
... etc ...
-
arrivalRate: 50
duration: 2

Environments
Typically we'd want to re-use a load testing script across multiple enviroments (e.g. dev , staging , and
production ) with minor tweaks. That's what config.environments is for: a number of named
environments can be defined with environment-specific configuration.

For example, a typical use-case is to define multiple targets with di erent load phase definitions for each of
those:
config:
target: "http://wontresolve.local:3003"
phases:
- duration: 10
arrivalRate: 1
environments:
production:
target: "http://wontresolve.prod:44321"
phases:
- duration: 120
arrivalRate: 10
staging:
target: "http://127.0.0.1:3003"
phases:
- duration: 1200
arrivalRate: 20
scenarios:
- ...

Choose an environment on the command line with the -e flag; e.g.


artillery run -e staging my-script.yml .

The $environment variable


The name of the current environment (if set) is available in the $environment variable.

Example - printing the name of the current environment from a scenario:

config:
#
# config here
#
scenarios:
- flow:
- log: "Current environment is set to: {{ $environment }}"

Payload Files
It can often be useful to be able to inject data from external files into your test scenarios. For example, you
might have a list of usernames and passwords that you want to use to test the auth endpoint in your API.

Payload files are in the CSV format and Artillery allows you to map each of the rows to a variable name that
can be used in scenario definitions. For example:
config:
payload:
# path is relative to the location of the test script
path: "users.csv"
fields:
- "username"
- "password"
scenarios:
- flow:
- post:
url: "/auth"
json:
username: "{{ username }}"
password: "{{ password }}"

We tell Artillery to load users.csv file and make variables username and password available in
scenarios containing values from one of the rows in the CSV file

To import multiple CSV files "payload" may also be an an array:

payload:
-
path: "./pets.csv"
fields:
- "species"
- "name"
-
path: "./urls.csv"
fields:
- "url"

It's also possible to load a di erent CSV file depending on the environment you set with the
--environment flag. For example, you may want to load a di erent set of usernames/passwords to use
with an authentication endpoint when running the same test in di erent environments:

payload:
- path: "{{ $environment }}-logins.csv"
fields:
- "username"
- "password"

Options
order (default: random ) -- control how rows are picked from the CSV file for each new virtual user.
Set to sequence to iterate through the rows in a sequence (looping around and starting from the
beginning after the last row has been reached).
skipHeader (default: false ) -- set to true to make Artillery skip the first row in the file (i.e. the
header row)
delimiter (default: , ) -- if a delimiter other than comma is used, set this option to the delimiter
character
cast (default: true ) -- by default Artillery will convert fields to native types (e.g. numbers or
booleans). To keep those as string, set this option to false .
skipEmptyLines -- by default Artillery skips empty lines in the file. Set to false to include empty
lines.

Example

config:
payload:
path: "users.csv"
fields:
- "username"
- "password"
order: sequence
skipHeader: true
scenarios:
- # ... the rest of the script

Inline variables
Variables can defined in the config.variables section of a script and used in subsequent request
templates.

 Variables defined in this block are only available in scenario definitions, i.e. they cannot be used to
template any values in the `config` section of your scripts. If you need to dynamically override values in
the `config` section, use environment variables in conjunction with $processEnvironment .
config:
target: "http://app01.local.dev"
phases:
-
duration: 300
arrivalRate: 25
variables:
postcode:
- "SE1"
- "EC1"
- "E8"
- "WH9"
id:
- "8731"
- "9965"
- "2806"

The variables can then be used in templates as normal. For example: {{ id }} and {{ postcode }} .

Setting success conditions with ensure


When running Artillery in CI/CD pipelines, it can be useful to have Artillery exit with a non-zero code when
a condition is not met.

Latency
To check that the aggregate p95 latency is 200ms or less, and make Artillery exit with a non-zero exit code
if it's over 200, add the following configuration to your script:

config:
ensure:
p95: 200

min , max , median , p95 , and p99 may be set.

Error rate
The error rate is defined as the ratio of virtual users that didn't complete their scenarios successfully to the
total number of virtual users created during the test.

To make Artillery exit with a non-zero if the total error rate exceeded 1%:

config:
ensure:
maxErrorRate: 1
scenarios
The scenarios section is where one or more virtual user scenarios are defined.

A scenario is a sequence of steps that will be run sequentially which represents a typical sequence of
requests or messages sent by a user of an application.

A scenario definiton is an object which must contain a flow attribute and may contain a number of other
attributes.

flow - a "flow" is an array of operations that a virtual user performs, e.g. GET and POST requests for
an HTTP (/docs/http-reference)-based application.
name - allows to assign a descriptive name to a scenario, e.g.
"search for a product and get its details"
weight - allows for the probability of a scenario being picked by a new virtual user to be "weighed"
relative to other scenarios

Some Artillery engines will also support other scenario attributes, e.g. the HTTP engine (/docs/http-
reference) allows for scenario-level hooks to be defined.

Scenario weights
Weights allow you to specify that some scenarios should be picked more often than others. If you have
three scenarios with weights 1 , 2 , and 5 , the scenario with the weight of 2 is twice as likely to be picked
as the one with the weight of 1 , and 2.5 times less likely than the one with weight of 5 . Or in terms of
probabilities:

scenario 1: 1/8 = 12.5% probability of being picked


scenario 2: 2/8 = 25% probability
scenario 3: 5/8 = 62.5% probability

Weights are optional, and if not specified are set to 1 (i.e. each scenario is equally likely to be picked).

You might also like